Using Nix to Create Python Virtual Environments

Nix and Python logos

Nix is a great tool to set up development environments. It allows us to have simultaneous installations of various versions of tools—such as Python—required for our projects. This means Nix makes it easy to have Python 2.7 installed for one project, and Python 3.6 for another. Projects using the same Python version can have different Python packages.

Of course, Python’s VirtualEnv also enables us to do this. Nix, however, is more powerful. It can handle all our system’s packages; not just Python’s. This means it enables us to hold different versions of any dependency. For example, if one project requires a specific version of OpenCL and another project requires an incompatible version, VirtualEnv won’t help us. Nix will.

There are many Python packages, and to install one such package through Nix requires it to be available in the Nix package repository. Understandably, not all Python packages are packaged for Nix—and those that are, often are not the newest version, nor at some other specific version we require.

We can use Nix to provision a Python environment for our project that works similarly to VirtualEnv’s. We can then use pip to handle such per-project Python dependencies, allowing us to grab Python packages directly from the regular Python package repositories without going through Nix. This also allows us to quickly get to work with others’ Python projects that are not set up to work with Nix.

Continue reading “Using Nix to Create Python Virtual Environments”

Closed-Form Rocket Fuel Requirements

Previously we calculated the fuel requirements of rockets to reach escape velocity. In that calculation, we did not take into account the effect of gravity during the fuel burn and underestimated the fuel requirements. We improved on this through a simulation better approximating reality. This simulation takes into account the force of gravity during the burn, as well as that force weakening as the rocket increases its distance from Earth, and takes into account the necessary escape velocity decreasing as the rocket’s distance to Earth grows. On this second post, Basker V commented that a closed-form solution can be found when the burn time is sufficiently small, allowing the rocket’s distance to Earth to be considered constant during the burn. In effect, this enables us to ignore both the diminishing force of gravity and decreasing escape velocity. Let’s find this closed-form.

Continue reading “Closed-Form Rocket Fuel Requirements”

&SPL – Creating a simple imperative programming language

SPL sample code

&SPL is an acronym for Simple Programming Language, and is a C-like imperative programming language, with classes, lists, and tuples. For example, the following code produces the Fibonacci sequence.

#include "stdlib/std.spl"

// Output the Fibonacci sequence
main ()
{
    var t1 = 0;
    var t2 = 1;

    while(True) {
        println(t2);
        var h = t1 + t2;
        t1 = t2;
        t2 = h;
    }
}

The language is formally defined with a context-free grammar and typing rules. In this post an implementation of an &SPL source code compiler implemented in Haskell will be discussed. The compiler compiles the source code into assembly for a Simple Stack Machine [1] (http://www.staff.science.uu.nl/~dijks106/SSM/) in a four-part process of lexing, parsing, typing and code generation. The source code is available on GitHub.

Continue reading “&SPL – Creating a simple imperative programming language”

[1] A. Dijkstra, “Simple stack machine.”
[Bibtex]
@Manual{SSM,
title = {Simple Stack Machine},
author = {Atze Dijkstra},
timestamp = {2017-06-18},
url = {http://www.staff.science.uu.nl/~dijks106/SSM/},
}

Creating Multiple Custom User Types Through Inheritance in Django

Django Logo

When developing applications in Django, the need might arise to customize the user model. Specifically, you might want to create different types of users. In my case, I’m interested in creating a person user and a kit user. A person user can own multiple kit users, and both need to be able to authenticate to access an API. Luckily, Django’s authentication system is very flexible, and there are multiple ways to achieve this goal.

The standard way to implement this is to stick with the default user model, django.contrib.auth.models.User, and create a complex user profile. The profile adds the desired fields and behaviors for the various user types in a new model, and links to the model through a field reference. This can get fairly complex quickly. It is especially difficult to express ownership of kits by users, without allowing ownership of users by users. Here, we will see how we can implement this using inheritance.

Continue reading “Creating Multiple Custom User Types Through Inheritance in Django”

ASUS PCE-AC68 Blue Screen of Death

Since building a new computer, my Windows 10 installation would sometimes crash. The error reported on the blue screen of death would always be page fault in nonpaged area. Interestingly, the system would be fine for months and then suddenly crash a few times per day — always when using Deluge, a torrent client. After some testing, it turned out that downloading a torrent with Deluge would reliably crash the system within an hour.

Debugging the crash minidump with Windows Debugger (WinDBG), it turned out the driver bcmwl63a was likely at fault. This driver is developed by Broadcom for 802.11 wireless network adapters. The driver was being used for my ASUS PCE-AC68 network adapter.

I was using version 7.35.338.0 of the Broadcom drivers for the adapter, supplied through the ASUS product utility version 2.1.1.5 that is available on the ASUS support website. The issue can be fixed by upgrading to version 7.35.351.0 of the Broadcom drivers, which ASUS supplies in version 2.1.4.3 of their utility. The issue can likely also be fixed by downgrading to version 6.30.223.228 (utility version 2.0.8.8), but I have not tested this. See this thread for more information.

First uninstall the old driver through Windows Device Manger (and uninstall the utility through the Control Panel, if installed). Then, instead of installing the new utility, I would recommend installing only the new driver (supplied in the same zip-file as the utility). This can be done manually in Device Manager by updating driver software for the PCE-AC68. Note that after uninstalling the old drivers, the adapter might show up as an unrecognized device, so be sure to check that if the PCE-AC68 is no longer present in the device list.

 

Hibernation and Hybrid Sleep on Ubuntu 17.04, Gnome 3.24

I was having trouble using suspend, hibernate and hybrid sleep functionality on Ubuntu 17.04 using Gnome 3.24. The troubles started immediately after installing the uswsusp package:

apt-get install uswsusp

I was hopeful and typed systemctl hibernate, but the system got stuck on Snapshotting system.

After a hard shutdown, the system was no longer bootable. It would only boot in recovery after waiting for 5 minutes on /scripts/local-premount.

These issues were caused by having an encrypted swap file, but soon other issues surfaced, rooted in shaky support for hibernation in both Ubuntu and Gnome.

So, here’s a guide to how I set up hibernation.

Continue reading “Hibernation and Hybrid Sleep on Ubuntu 17.04, Gnome 3.24”

mIRC Smart Join/Part Filtering

Though internet relay chat (IRC) is old technology, many people still use it daily. One of the most popular IRC clients for Windows is mIRC. This client is impressively feature-complete. However, it does not offer a way to satisfyingly solve one of the grievances of many IRC users: the flood of join and part messages in big channels, caused by the constant stream of people coming in and leaving. mIRC only offers the option to either show or hide all join and part messages completely; but does not allow for any fine-tuning.

What I was looking for was a way to show or hide join and part messages based on the size of the channel. If a channel has only few people, I’d like to see all joins and parts. If, however, a channel has many users (e.g., more than 25), this quickly becomes a nuisance. In addition, even in big channels, I’d like to see all joins and parts of people who were recently active in the channel, so as not to accidentally reply to someone who has left in the meantime.

With this specific need, I created a solution, which I have made available on github. It can be added to mIRC by loading it as a script (Tools -> Scripts Editor -> File -> Load). Configure it by right clicking in a chat window, and select Join/Part Filter -> Settings.

Smart join/part filter
The smart join/part filter setting screen

Programming this solution using mIRC’s scripting language was definitely a fun project: the scripting language is quite basic with little documentation, and as mIRC’s interpreter offers hardly any assistance when things go wrong, it quickly turns into a puzzle. However, it’s also quite powerful and allows you to access and modify many things within mIRC.

A Bot to Log Discord Voice Events

Discord is a great platform for text and voice chatting. However, one feature has been seriously lacking for a while now: the ability to see a log of people joining and leaving voice chat rooms.

Luckily, Discord provides a way to create bots. These bots have access to a variety of events occurring on a Discord server, including voice chat events. Of specific interest to us, bots can track events of people joining and leaving voice chat rooms.

I set out to create a simple bot able to log these events to a specific Discord channel. I programmed the bot in Python, using the discord.py library (v0.16.7). My code is open source and available on github. See the bot in action in the video below:

If you’d like to use this bot, feel free to use the code provided; you’ll probably need to create a new developer application on Discord and turn it into a bot user, and read the read-me.

Machine Translation Turing Test

Machine Translation
Will computers ever reach the quality of professional translators?

The U.S. government spent 4.5 billion USD from 1990 through 2009 on outsourcing translations and interpretations [4]. If these translations were automated, much of this money could have been spent elsewhere. The research field of Machine Translation (MT) tries to develop systems capable of translating verbal language (i.e. speech and writing) from a certain source language to a target language.

Because verbal language is broad, allowing people to express a great number of things, one must take into account many factors when translating text from a source language to a target language. Three main difficulties when translating are proposed in [6]: the translator must distinguish between general vocabulary and specialized terms, as well as various possible meanings of a word or phrase, and must take into account the context of the source text.

Machine Translation systems must overcome the same obstacles as professional human translators in order to accurately translate text. To try to achieve this, researchers have had a variety of approaches over the past decades, such as [3, 2, 5]. At first, the knowledge-based paradigm was dominant. After promising results on a statistical-based system ([2, 1]), the focus shifted towards this new paradigm.

Continue reading “Machine Translation Turing Test”

[1] P. F. Brown, V. D. J. Pietra, S. D. A. Pietra, and R. L. Mercer, “The mathematics of statistical machine translation: parameter estimation,” Computational linguistics, vol. 19, iss. 2, p. 263–311, 1993.
[Bibtex]
@article{Brown1993,
title={The mathematics of statistical machine translation: Parameter estimation},
author={Brown, Peter F and Pietra, Vincent J Della and Pietra, Stephen A Della and Mercer, Robert L},
journal={Computational linguistics},
volume={19},
number={2},
pages={263--311},
year={1993},
publisher={MIT Press}
}
[2] P. F. Brown, J. Cocke, S. D. A. Pietra, V. D. J. Pietra, F. Jelinek, J. D. Lafferty, R. L. Mercer, and P. S. Roossin, “A statistical approach to machine translation,” Computational linguistics, vol. 16, iss. 2, p. 79–85, 1990.
[Bibtex]
@article{Brown1990,
title={A statistical approach to machine translation},
author={Brown, Peter F and Cocke, John and Pietra, Stephen A Della and Pietra, Vincent J Della and Jelinek, Fredrick and Lafferty, John D and Mercer, Robert L and Roossin, Paul S},
journal={Computational linguistics},
volume={16},
number={2},
pages={79--85},
year={1990},
publisher={MIT Press}
}
[3] D. A. Gachot, “The systran renaissance,” in Mt summit ii, 1989, p. 66–71.
[Bibtex]
@inproceedings{Gachot1989,
title={The SYSTRAN renaissance},
author={Gachot, Denis A},
booktitle={MT SUMMIT II},
pages={66--71},
year={1989}
}
[4] D. Isenberg, Translating For Dollars, 2010.
[Bibtex]
@misc{USSpend,
author = {David Isenberg},
title = {{Translating For Dollars}},
howpublished = "\url{http://www.huffingtonpost.com/david-isenberg/translating-for-dollars_b_735752.html}",
year = {2010},
note = "[Online; accessed 2-October-2013]"
}
[5] P. Koehn, F. J. Och, and D. Marcu, “Statistical phrase-based translation,” in Proceedings of the 2003 conference of the north american chapter of the association for computational linguistics on human language technology-volume 1, 2003, p. 48–54.
[Bibtex]
@inproceedings{Koehn2003,
title={Statistical phrase-based translation},
author={Koehn, Philipp and Och, Franz Josef and Marcu, Daniel},
booktitle={Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1},
pages={48--54},
year={2003},
organization={Association for Computational Linguistics}
}
[6] A. K. Melby, Some difficulties in translation, 1999.
[Bibtex]
@misc{TTT1999,
author = {Alan K. Melby},
title = {{Some difficulties in translation}},
howpublished = "\url{http://www.ttt.org/theory/difficulties.html}",
year = {1999},
note = "[Online; accessed 2-October-2013]"
}