Hermes – A Textboard

Internetless in London

Something strange happens when you’re suddenly disconnected from the Internet for a long period of time: You find other things to do. After cleaning the bathroom and tidying my room twice, beating some video games on my backlog, and somehow losing offline access to Spotify (thanks Spotify), it was time for something useful. It was time to watch a lot of Japanese drama.

I’d already watched Densha Otoko before, but I remembered it being a particularly fun romantic romp, incorporating Japanese internet culture into the plot as a major driving force. Amazingly, it’s based on a true story. Well, as true as a story from the Internet can be. The plot revolves a nerd called Yamada, who is far more comfortable in the company of his excessive collection of anime figures than around people. One fateful evening on the way back from an otaku themed event, Yamada encounters a young woman being harassed by a drunk passenger. Mustering all of his courage, he steps in and promptly gets beaten up by the drunk. Fortunately, help arrives and Yamada is saved. The young woman asks for Yamada’s address so she can send him a thank you present.

Once home, Yamada leaps on to his favourite Internet textboard, Aladdin Channel, and writes a post describing his experience and asking for advice. The plot continues with Yamada, dubbed the Train Man (Densha Otoko) by the textboard, putting Aladdin Channel’s advice into action, and attempting to win the woman’s heart.

While watching the show, I figured I could do something productive at the same time. Hermes is my first attempt at a proper, fully-featured Django application. Of course, Hermes is a textboard.

 What’s a textboard?

If you’ve ever been on an Internet forum, you’re already familiar with a different version of the textboard. Textboards are piece of software that allow an administrator to create a number of boards, on which users can post threads and replies to those threads. The biggest textboard out there is 2channel (NSFW), on which the original ‘real life’ Densha Otoko story was posted. It boasts over 600 boards covering a huge range of topics, and a healthy user base.

With textboards, there are two main differences from your standard forum software:

  • Users do not need to register to use a textboard. Indeed, registration likely is not an option at all. Users post either anonymously or with a nickname they provide. These nicknames are not secured in any way, meaning any user could take any nickname. Users who want their posts to be attributed to them and only them can post with a tripcode, a secret password known only to them that generates a hash that is attached to their posts.
  • The content of the first post of each thread is shown on the board page, as well as the content of the latest three or four posts in that thread. This differs from most forum software, which will only show a list of thread subjects on a given board’s page. Because of the way the content of each thread’s first is shown, subjects for each thread are not mandatory on textboards.

In addition to the above, threads are pruned from the database after they see a certain amount of inactivity. Hermes, for example, is configured to delete the oldest thread if the total number exceeds fifty on a single board.

If  you add images to the mix, you get an imageboard. It’s likely you might have heard more about these, such as the infamous 4chan and its Japanese counterpart 2chan (both NSFW).

Perhaps due to in part of the transient nature of posts and identity on these text- and imageboards, the culture that develops is that of a hivemind. Voices that try and rise above the others by using a name or tripcode are often berated and ignored. Of course, anonymity comes with its own problems. Unfortunately, I’m not an anthropologist, so I can’t really go more into that.

Why Hermes? And why aren’t you spelling it right?

‘Hermes’ is the name given to the female protagonist of Densha Otoko by the denizens of Aladdin Channel after Yamada receives a set of Hermès teacups from her. The Internet was down so I couldn’t Google around for cool names to give my textboard, so Hermes ended up being the name of choice. The name in the drama isn’t pronounced with the lovely accent on the e, so I elected to not use it.

Django Unplanned

My lack of Internet connection put some limitations on what I could use for this project. Sure, I could enable tethering on my phone, but to pull down entirely new software packages? That would get messy for my phone bill. Django happened to be installed on my Virtualbox after an abortive attempt at another web site, so I figured ‘hey, why not use Django?’

Reading the tutorials on my tiny S3 phone’s screen was a less than exciting experience, but with just a couple of hours work I had a super basic textboard working. Django makes CRUD applications very easy to implement. It only took a few dozen lines of code to implement the initial database structure, and the templating system for the front-end is pretty intuitive when it came to the front-end.

Heroku

The demo instance of Hermes (Hermes Channel) is deployed on Heroku, an infrastructure-as-a-service platform geared towards people like me who hear the word ‘infrastucture’ and fall asleep, only to be greeted by dark nightmares of configuration files and servers. It’s so easy to use: Once your Django app is ready and you’ve got a file describing your prerequisites you need, you can just commit to a Git branch hosted on Heroku, and they deal with the deployment automatically.

It also has the added benefit of being free: You get a free 10k row Postgres database, and 750 free hours per month to run your dynos. An Heroku dyno is basically a Linux instance that your application (or worker jobs) will run on. When you deploy an application to Heroku, it is deployed to all your dynos and started automatically. If you have just one dyno, the free allowance is enough to last the entire month! Of course, if your application has high traffic, it is likely you’ll need more than a single dyno to support it. But then again, if you have that much traffic you’re probably earning enough money to support beefing up your Heroku too, so…

The Demo!

Hermes is effectively feature-complete, so why not check out the demo? It’s mobile friendly! It’s easy to make posts too. If you’re interested in knowing how it all ticks, the source code for Hermes is available on Github.

Okay, fine, you can have a screenshot.

This feels familiar somehow.

This feels familiar somehow.

Check it out! Hermes will be released under the GPLv3. Releasing the software was always the main point of the project, the fact there’s a demo instance is just a pleasant bonus! Just follow the instructions on the Github page for installation, and you’ll be away (provided you know a bit about how Django works).

 

Speare2Brain

The predecessor project of Speare2Brain, Brain2Speare, took all of a day to implement. It took another day to spruce up, but the fact was: After a single day of development, nearly every Brainfuck program could be successfully transpiled to Shakespeare[1]. The same cannot said of Speare2Brain. Also, spoilers, it only partially works.

What is hell is a parser generator?

The result of the original Shakespeare project was a Shakespeare-to-C transpiler called spl2c. This is impressive in itself, and is mostly powered by GNU Bison and flex (a parser generator and lexical analyser, respectively). Flex is responsible for taking an spl file and tokenising it for consumption by a parser generated by Bison. While parsing, the program will construct C code that corresponds directly to a set of spl tokens. There’s no need for multiple passes when building the C code, because the generated C is simply a translation of the Shakespeare. For example, the command ‘Romeo: Remember thyself’ is tokenised by flex as ‘REMEMBER SECOND_PERSON_REFLEXIVE‘, which a parser generated by Bison will consume. The parser will then output code like ‘push(second_person, value(second_person))‘. The functions and constants used by the generated code comprise the remainder of the Shakespeare project: In addition to the code generated by spl2c, one must link against libspl and include spl.h when compiling to a binary.

You transpile me right round baby, right round

Of course, when I started the project I didn’t know any of this. I took a course on compilers back at university that I must have mostly sleepwalked through (although I remember doing okay?). Was there a way to take advantage of the work already done by the Shakespeare project? Of course! The hard part of lexical analysis was already done, and the parser generator would make a a great jumping off point. If I modified the parser generator, I could possibly directly output the Brainfuck instead of C! Except… Bison’s parser generation code is heavily C based. Have you ever tried working in C? I have no doubt that it would have been possible to bend Bison to my will and perform the complete transpilation as part of the parsing, but it was beyond my power and patience. If I was going to finish this project in a reasonable time-frame, I would have to work in a language I was more comfortable in.

So why not get all meta about it? I eventually decided to modify the parser generator to output something that wasn’t C. Instead, it would output a brand new language! That language came to be, and was called ‘Not-Shakespeare Programming Language’, or NSPL. By tearing out most of the good stuff from the C generation bit of spl2c and replacing it with my own horrific code, I was able to create a new program that I imaginatively named spl2nspl. This part of the project took an evening in itself to figure out and implement. Let’s compare the some of the output from spl2c and spl2nspl real quick.

C NSPL
activate_character(30, hamlet); assign(16, second_person, 2*2*2*2*2*2*(-1)); assign(18, second_person, int_sub(17, 2*2*2*1, value_of(18, second_person))); char_output(18, second_person); activate,hamlet, assign,const,-64,end_assign, assign,sub,const,8,value_of,second_person,end_sub,end_assign, output

Yep. It’s the same thing, but written out such that it can be more easily parsed by the real workhorse of the project: A Python script named… Wait for it… nspl2bf. This is the main part of Speare2Brain that I’d like to talk about.

To be or not to be? If so, let us proceed to scene III.

Brain2Speare was a relatively simple project for many reasons. The biggest reason is as follows: It is very easy to rewrite a low-level instruction as a higher-level instruction. After all, you can translate Brainfuck symbol-for-symbol into C code, and Brain2Speare proved that it was also possible to find a similar mapping between Shakespeare and Brainfuck. What about the the reverse? That is to say, translating a higher-level language into a lower-level language. This is what real compilers and linkers do all the time: Binary files are as low-level as you can get.

Shakespeare-to-Brainfuck (or more accurately, Not-SPL to Brainfuck) was significantly more difficult to design and implement. Brain2Speare was based entirely on the idea that we could use two Shakespeare characters to represent the Brainfuck memory array. Let’s dive in to how the memory array is laid out in Speare2Brain. Once again, p is a pointer into the Brainfuck array. Shamelessly ripped from the explanation in the Github repository, and then modified for accuracy:

p[0] - Copy Register (Copy)
p[1] - Result Register (Result)
p[2] - Loop Termination Register (Loop)
p[3] - Character Value Retrieval Register (Retrieve)
p[4] - Temporary Register (Temp)
p[5] - Right Register (Right)
p[6] - On Stage 1 Register (OS1)
p[7] - On Stage 2 Register (OS2)
p[8] - Active Character Register (Active)
p[9] - Inactive Character Register (Second)
p[10] - Left Register (Left)
p[11] - First Character Register (First character's register)
p[12] - Second Character Register (Second character's register)
...

The registers are named after what they are used for the most. In reality, all the registers below p[6] are temporary and cannot be relied upon to be clean at any time. Internally, nspl2spl keeps track of the offset of each register and the ‘current’ location of the Brainfuck pointer in a class called MemoryLayout. In addition to the normal registers, we also have a number of character registers. These represent the character from the original Shakespeare, and are supposed to hold the same values and perform much of the same function as the character in the C equivalent.

I’ll go through what each register is used for, but be warned: You’ll need to know a bit about Brainfuck for this, because this is more about the hard implementation and less about the transpilation process.

Memory Registered

Time for some more shameless copying from the documentation:

  • Copy – In BF, the simplest way of moving an unknown-until-runtime value from one cell to another involves looping on the source cell; decrementing the value of that cell, moving to the destination cell, incrementing that cell, then moving the pointer back to the source. In this fashion, the source cell will be emptied and the destination cell will gain the value that the source cell originally held. However, this destroys the source cell. The Copy register allows for copies between cells by not only incrementing the value of the destination cell inside the loop, but also that of the Copy register. Once the source cell is emptied, we move the value in the Copy register back into the source cell.
  • Result – Multiple uses, but one notable use inside nspl2bf is as an indicator for if-else statements. We set the result register to 1 before moving the pointer to another register and attempting to enter one of two blocks, only one of which we want to execute. If we enter the first block (the ‘if’ block) successfully, we immediately decrement the Result register. After we leave the if block, we test Result and only enter the else block if it is still non-zero; that is to say, we did not enter the first block.
  • Loop – In nspl2bf, it is very important that the pointer is not moved manually. You use functions in the MemoryLayout class to return movement commands, do your BF magic, then reset the pointer. However, this puts a constraint on where the pointer must be at the end of a [] block: It has to be at the same place as it would have been if the block wasn’t entered at all, otherwise we won’t know how many < commands are needed to reset the pointer. This means we have a problem if we need to terminate the loop but we know that the value in the register we used to enter the loop won’t necessarily be zero. To get around this, we use the Loop register: Move the value of the ‘entering’ register into Loop, move the pointer to said ‘entering’ register, allow the loop to terminate, and then move Loop back into the original register.
  • Retrieve – This is horrible to explain. In nspl2spl we store the offsets of character memory cells in other registers named Active and Second. Retrieve is used to hold these offsets when forcing the memory pointer to the character’s cell via some really twisted construction. It’s only named Retrieve because it was created out of necessity while writing the assign transpilation code. It serves as any other temporary register otherwise.
  • Temp – It does whatever it needs to do. It’s not the register we deserve, but it’s the one we need.
  • Right – Holds the value for the right and unary argument in binary and unary expressions respectively.
  • On Stage (OS1, OS2) – The On Stage registers contain the memory offsets of the characters currently on stage. They’re necessary because when we ‘activate’ an On Stage character, we need to put the inactive character into the Second register. To do this, we need to know which characters are on stage, hence the OS registers.
  • Active – This holds the offset of the active on stage character. More on this below.
  • Second – Holds the offset of the character that is on stage but NOT active. This is very useful, because many commands like assign operate exclusively on the inactive character. Having their offset stored simplifies things, but actually accessing the value at the offset is something of a pain. However, because we can construct some really twisted Brainfuck to let as get that memory pointer in the right place.
  • Left – Holds the value for the left argument in binary expressions. It occupies this part of memory because it is secretly the first character, which means it gets its own stack. This is required because nspl2bf evaluates binary expression trees from left-to-right, meaning the value in the left register can get clobbered when calculating the right value. To avoid this, Left has its own pseudo-stack for storing evaluated expressions.
  • Character registers + stacks – These hold the value that each character currently, well, holds. Each character has a stack counter cell n cells after their initial offset, where n is the number of characters in the original Shakespeare. The next cell n spaces after marks the bottom of the character’s stack, and every nth space after that is another possible value of their stack.

Do you bite your thumb at me, sir?

A couple of high level examples for what the transpiler does: Each token read by nspl2bf from an nspl file generates Brainfuck code for manipulating the above registers. For instance, the activate token will cause the parser to grab the next token (the name of a character), then output the Brainfuck for copying that character’s offset into the Active register. Under the assumption that this character is already on stage, the other character who is on stage will have their offset copied into the Second register. Why? Because that’s what the original C would have done: If a character is activated, the other character on stage is inactive. Certain other tokens use the active or inactive character for their functions, so it’s important that we know which characters are both on stage and active.

The assign token (a true monster to implement) first clears the Result register before parsing all the tokens between itself and end_assign. The tokens between these two are expression tokens that represent one of the following: Constant values, unary functions, and binary functions. After each of these are applied, we expect a value to be stored in the Result register that is then copied into the memory offset pointed to by the Second register.

At a high level, it’s quite easy to describe what each function does. However, writing the Brainfuck to actually do all that? It’s horrible. It’s really, really hard (not for a good programmer, but yeah). Especially for things like the GOTO statements and the character stacks that we loved so much in Brain2Speare. In fact, those are so hard, they’re not even implemented. That’s right, Speare2Brain isn’t finished. However, it has passed a major development milestone: It successfully transpiles the Hello World Shakespeare program into Brainfuck. Even better, the Brainfuck itself actually runs and prints ‘Hello World!’. There’s one tiny little caveat, but I don’t really care:

nifrith@Phoenix> wc -l examples/hello.spl
89 examples/hello.spl
nifrith@Phoenix> wc -m examples/hello.bf
22739 examples/hello.bf

Yeah, the end result is huge, but we’re not writing gcc here. But nevertheless:

OhBaby

Oh baby.

Speare2Brain deserves way more explanation, especially more details on the transpilation itself. I’ll write more about it in a follow-up post: We’ll dive into the real crunch that is the mapping between nspl and Brainfuck. In the meantime, check out the Github repository for the project! If you want to see a bunch of example nspl, just clone the repository and run make. It should generate all the example nspl files along with some non-functioning Brainfuck for every file except hello.spl.

Footnotes

[1] An exception until recently included programs that relied on a wrapping cell implementation of Brainfuck. Until I wrote Speare2Brain, I didn’t even realise such implementations existed, but they end up making many, many otherwise difficult tasks much simpler.

Curses! – A Post That Ends Up Being About Capturing Ncurses Output And Converting It To HTML

So I had an idea the other day! I started to implement it, but it turned out it was already done way better elsewhere, so I dropped it. But let’s talk about it anyway, because it’s not the worst idea ever and I had a fun experience as a result.

For some reason, I was under the impression that a really good web-based Interactive Fiction interpreter wasn’t available. I have no idea why I held this idea, because I had been using Parchment to play Z-Machine games online for the past couple of days. Maybe I figured there wasn’t a good online Glulx interpreter, but I was wrong about that too. In any case, the idea was simple: Provide a reasonable method of playing Interactive Fiction online with near-perfect emulation of how it would feel when played on a native interpreter.

The method I chose was chosen because I am lazy. It was literally to be a bunch of different things I found online all cobbled together. The fundamental plan was to run a Linux Z-Machine interpreter for each user, take ‘screenshots’ of the terminal, convert this screenshot to HTML and serve it to the user via a web interface. Input would be sent down to the interpreter from the browser, ‘screenshots’ get sent back up. This would have meant the user experienced ‘lag’ when typing, but I was just pleased that I thought I struck upon a novel method of delivering an authentic native Z-Machine experience in the browser.

Rather than talk about why that was all a terrible, terrible idea, let’s focus on the journey and not the aborted destination. While the idea didn’t pan out, I did learn a thing or two about the terminal and ANSI codes. In fact, I even found and fixed a bug in tmux! The fix hasn’t been accepted as a patch yet, and it might not be because it’s not the cleanest solution to the problem… But I did discover a bug, so yay!

So how did I end up doing that? Well, it turns out that the very first step of the journey was a doozy. Capturing output from the terminal is not usually very difficult. I was using a Z-Machine interpreter called Frotz to do the heavy-lifting of running the IF games. Naturally, my first attempt was redirecting the output of Frotz to a file and finding a tool to convert the ANSI to something more web-friendly. That’s when I came across ansi2html.sh, which didn’t work at the time for a very particular reason. Frotz uses something called ncurses to draw its screen. Rather than scroll the terminal when new text has to be printed, it prints over the existing space by using special terminal control codes called cursor movement instructions. These instructions are used by ‘printing’ them to the output. When one of these characters is ‘printed’ by the terminal, the cursor is moved according to the instruction rather than a new character being printed. After being moved, new text is drawn over the text that was previously present in that spot.

This means that Frotz doesn’t output lines delimited by newline characters. Instead, it prints out one very long line that uses these cursor instructions to move the cursor around and redraw over already printed characters. This confuses ansi2html, which tries very hard but eventually can’t keep up with the complicated sequence of cursor movements. This explanation doesn’t do the process justice, so here’s a few pictures. First up, here’s a picture of what Frotz looks like when it’s running (or if the output file is printed using ‘cat’):
Frotz

That looks pretty standard, right? Here’s what the underlying ANSI looks like:
ANSI Horror

Wow. Interestingly, you can ‘cat’ this file and the output looks exactly the same as the first screenshot (Technically speaking. You would have to strip out a screen-clear instruction first). So ‘cat’ can handle it, but ansi2html.sh produces this:
Nice try, ansi2html

It’s a good effort, but it’s clear that ansi2html can’t do this. Other tools purported to be able to convert ncurses ANSI output to something more usable, but I tried them to no avail. I needed some sort of pre-processing step: To convert this mess of ANSI characters into a more easily manageable mess of ANSI characters. Enter tmux.

Tmux is a terminal multiplexer, just like ‘screen’. It has one advantage over ‘screen’ that I was interested in: It can capture coloured ‘screenshots’ of the terminal and write them to a hardcopy. A hardcopy is basically just a screen capture, like the output file from earlier but stripped of cursor control characters and the like. Tmux generates this by examining each individual character in its viewing pane and deducing which ANSI flags would have to be set to produce that character. In effect, tmux produces a perfect facsimile of terminal output, minus any crazy terminal control characters. Generating the hardcopy isn’t hard. I opened a terminal and booted up tmux. Once within tmux (which again, functions very similarly to ‘screen’), I started Frotz and used another terminal to send the commands to the tmux process, causing it to capture its current viewing pane to a buffer and then saving that buffer to a file called tmux.hardcopy:
tmux capture-pane -eJ ; tmux save-buffer tmux.hardcopy

So did it work? We should be able to ‘cat’ the hardcopy and get back the same image as the first one.
So close

Okay, but where did the colour go?! I can see it in the header, but it suddenly disappears. Well, ignoring that, how does ansi2html perform with the hardcopy?
Nearly there

That’s so much better! But the colour is still missing. What gives? Well, it turns out that tmux has a bug which incorrectly sets certain flags. If certain attributes change between characters (in this case, the ‘reverse colours’ attribute), a special character is printed which resets all attributes. The remaining attributes are then re-added via their own special character instructions. However, the colour instructions are not refreshed like the other attributes, so suddenly we’re missing all the colour from the output from that point on because tmux believes it only has to print the colour changing code once at the start of the hardcopy, rather than after every attribute reset. So I wrote a patch for tmux! It’s one of my first experiences with contributing to open source.

As I was writing this, one of the tmux maintainers got back to me regarding my patch: It’s good, but it’s not quite there. It has its own problems, but we’re working through it to get a proper fix together. Hopefully no one will ever actually need the patch above!

Anyway, shortly after finding/fixing the tmux bug, I realised the project as it was envisaged was fairly pointless. However, I did get to find a real-life bug in something open source! And I got a blog post out of the whole experience too. And of course, this little post wouldn’t be complete without a final screenshot of ansi2html finally taking on an ncurses program and winning:
Victory!

Strictly speaking, I had to modify ansi2html to get the reverse colours working correctly. You can find my modified version at this gist here. But yeah! There you go: How to capture output from programs that use the ncurses library and convert them to html! Looks like something good came out of the journey after all. Maybe I’ll talk about how I tracked down the tmux bug next time?

Rogue Detective Part 2 – Curses!

Last time, we covered the rather ridiculous choices that led up to my decision to write a roguelike from scratch. Now, we’ll cover the first steps in the creation of Rogue Detective, and indeed, most games I write.

After the standard step of creating a new repository for source control comes the creation of the first source file. This is usually the most daunting part of development: Opening up your editor and seeing a completely blank file that needs to be populated. Step one: Save this blank file as ‘main.py’. Great, we’re half-way to finishing the game, but there’s no time for a break quite yet.

I already knew we’d be using the curses library, so a good first step would be to import it. Additionally, I wrote a barebones do-nothing main function and uses a curses utility function to call it. At this point, here’s the totality of what I had:

import curses

def main(stdscr):
    pass

if __name__ == '__main__':
    """Handles all the nasty stuff to do with curses set-up + tear-down"""
    curses.wrapper(main)

As part of the wrapper function, the main is called and passed a variable called stdscr. This is a curses Screen object. This is our canvas upon which we’ll draw our player character, NPCs; the world, basically. This will be passed to the object that controls the world, so it can also control the canvas. Calling the function via the wrapper serves a couple of purposes:

  1. The wrapper handles all curses set-up and tear-down, as mentioned in the docstring. It will initialise curses and set up various desirable cursor attributes such as disabling echo and enabling instant keyboard inputs. After the passed in function has ended, it shuts down curses gracefully.
  2. In case of an unhandled exception in the called function, the wrapper will catch it, shut down curses correctly, and then re-throw the exception. This prevents such a crash from destroying your terminal settings.

I went in to this particular stretch of work with the goal of having my character (an @ symbol) move around the screen controlled by the arrow keys. With the program successfully bootstrapped and in a ‘working state’, it was time to accomplish that goal.

I wanted to keep the main function as lightweight as possible, leaving it as only the entry point to the application and nothing more. I accomplished this by writing a Game class which the main would instantiate and then call a function of. In this case, the function is called mainLoop, and as you might expect, it is responsible for controlling the game loop itself. Before we get in to that, let’s examine the members of the Game class.

class Game:
    """The game logic itself. The loop and input handling are here."""
    def __init__(self, screen):
        """Create the screen, player, assets."""
        self.screen = screen
        self.running = True
        self.player = Player()

So all we have right now are variables for storing the aforementioned Screen instance, another for storing the state of the game (running, not running), and one containing a currently mysterious Player object. How about that mainLoop I mentioned?

    def mainLoop(self):
        """Run the game while a flag is set."""
        while (self.running):
            self.logic()
            self.draw()
            self.handleInput()

As long as the game is running, we run all the logic for the game, draw the resulting position, then wait for player input.

    def logic(self):
        pass

No game logic exists at this point, so logic is an empty function.

    def draw(self):
        """ Draw it all, filling in the blanks as necessary """
        self.screen.erase()
        player = self.player
        self.screen.addstr(player.y, player.x,
                           player.character, curses.color_pair(1))
        self.screen.move(player.y, player.x)
        self.screen.refresh()

draw actually does something useful. It clears the previous screen, grabs the Player object, draws it in the right place, and finally moves the cursor to the player’s position. That last step is one of the many niceties of a console-based roguelike: if we don’t put it directly on the player character, there are two problems:

  1. If we don’t uniquely colour the player character, the player (user, not game object) won’t be able to identify his character if multiple objects share the same symbol.
  2. If we don’t put the cursor on the player character, it has to go somewhere else. And where else would we even want to put it? It needs to go somewhere. Because of problem one, it makes sense to place it on the player character when possible.
    def handleInput(self):
        """ Wait for the player to press a key, then handle
            input appropriately."""
        character = self.screen.getch()
        if character == ord('q'):
            self.running = False
        elif character == ord('h'):
            self.player.move(Direction.LEFT)
        elif character == ord('j'):
            self.player.move(Direction.DOWN)
        elif character == ord('k'):
            self.player.move(Direction.UP)
        elif character == ord('l'):
            self.player.move(Direction.RIGHT)

Given the turn-based nature of the game, there’s no need to do anything particularly clever like pump event queues or have time-out functions. We explicitly wait for the player to input a character via a Screen function, check the result against the list of characters we’re expecting, and perform the associated action if we get a match. In this case, we call one of the Player functions to move the character around the screen. You’ll also notice we pass in an enum. In Python, enums are usually just static class variables grouped in a class that becomes the enum’s name:

class Direction:
    """An enum for the possible movement directions"""
    UP = 1
    DOWN = 2
    LEFT = 3
    RIGHT = 4

In this case, nothing clever is going on: An enum of four elements in declared for the four directions of movement. Let’s cover the Player object now:

class Player(object):
    """The player object, containing data such as HP etc."""
    def __init__(self):
        """Initialise the player object"""
        self.x = 20
        self.y = 30
        self.character = '@'

At the point in development, the Player contains only information strictly required to render it: Its position on the screen (y, x) and the character we should use to represent it. This is used by the draw function in the above Game class. It has a single member function:

    def move(self, direction):
        """Move the player one unit in the specified direction"""
        if direction == Direction.UP:
            self.y -= 1
        elif direction == Direction.DOWN:
            self.y += 1
        elif direction == Direction.LEFT:
            self.x -= 1
        elif direction == Direction.RIGHT:
            self.x += 1

This gets called by the handleInput function, and moves the player in the given direction. I should probably mention that this code will let you move the player character out-of-bounds of the Screen object. This throws an exception and crashes the program. Additionally, you can’t move to the bottom-right most part of the screen, for a much more subtle reason: Curses will draw the @ symbol in the very last available space, but then attempt to move the cursor along to a location that doesn’t exist in the screen space. Both of these problems will be fixed later on.

At this point, we can revisit the main function and see how we incorporate this new Game object. There’s no surprises here:

def main(stdscr):
    """Initialises the Game object and basically gets out of the way"""
    stdscr.bkgd(' ', curses.color_pair(0))
    game = Game(stdscr)
    game.mainLoop()

We set the ‘background’ to be made up of blank characters. If we ever erase the screen, it is actually filled with the background character, as set by the bkgd function. We then instantiate and start the game.

That’s it! The actual code looks a little different for the first commit (there’s a superfluous class floating around, and some of the code has been changed here for clarity). How does it look? Well, take a look:

FirstCommit

In the future, we won’t cover whole commits. Indeed, we won’t cover each commit, but rather the functional milestones of the project. Hopefully you’ve gained a little bit on insight in to how a Python curses project starts life. Next time, we’ll look at the implementation of walls and other standard obstacles you tend to see in roguelikes.

Rogue Detective Part 1 – A Defective Rogue

Rogue Detective started off as an attempt at the annual 7 Day Roguelike competition, wherein one aspires to create a full roguelike in a mere seven days. Entrants are allowed to use previous work, provided the final game is a new roguelike unto itself.

This year, the competition fell in one of the two weeks of my holiday from work. Being one of the many proud staycationers so prevalent in England, this meant I would have the full week to develop a game! I heard about the competition one day in advance of its start, which gave me a little bit of time to start mulling over a few ideas I had.

At the same time, I started considering at my options regarding development. I already knew what my editor would be: Emacs, of course! I figured if I was going to pull this off in a reasonable time frame, I’d need to use a language I was comfortable developing in. That meant a choice between C(++) and Python. Given how I’d recently worked on a Python game, the language was still fresh in my mind. Indeed, for speedy development I am in general more comfortable in Python.

Editor sorted, language sorted.. Now I needed to consider what libraries might exist out there to help me. I vaguely remembered NetHack using the curses library for terminal stuff, so the first thing I did was check for Python bindings. Sure enough, a Python library for curses existed. At this point, I hadn’t even considered a graphical roguelike. I knew I had no chance to produce something pretty, so I would stick to ascii. I started work on getting an @ symbol wandering the void. This is not the most thorough approach to library selection, but yeah! Limited time means limited research.

I hadn’t quite realised what I was undertaking: A complete roguelike from scratch, save for terminal character manipulation. Given only a week, surely the typical entrant would have to take the whole time to finish something even remotely playable, eschewing work and a social life?

Meanwhile, my friend Ariff was considering entering the same competition. The first thing he did was check for the already existing roguelike libraries that could help. Of course! Why wouldn’t something so obviously useful exist?

For Python, there already exists the libtcod library. Rather than build everything from scratch, libtcod provides users with useful functionality such as built-in dungeon generation, pathfinding, line of sight algorithms.. And to top it off, the whole thing is portable: Rather than build off a terminal-based library, it uses the SDL library. So while programs now have to run in a non-terminal, there’s very little messing about with getting games working on multiple platforms AND with full colour!

So it’s a no-brainer, right? Using this library, I’d be able to get to work on the meat of my game idea immediately! Maybe this wouldn’t be such an impossible task after all.

Of course, I’d already gotten an @ symbol walking around.. And it would be a shame to waste ‘all that work’.. Long story short, I made the questionable decision to continue with my own ‘from scratch’ engine. This was probably due to a mix of hubris and ignorance. Sufficed to say, I didn’t finish in seven days. It turns out that unless you’ve implemented things like path-finding and field of view before, they can be more than a little time consuming!

So the dream of a roguelike in seven days was dead on arrival. But this wouldn’t kill the dream of a roguelike in totality. Over the past month I’ve continued to work on my little game, and over the next few posts I’d like to go through some of the basics of roguelike development using my repository’s history to guide us. It will definitely not be a guide on the proper usage of source control (one branch for life!), but I think maybe it’ll be useful for me to solidify the process and maybe useful for people reading if they want some insights on how a newbie might write a roguelike.

Getting Virtual with Emacs (Part 3)

Last time, we talked about the two emacs extensions I use to emulate my most used features of Sublime Text. Even a cursory look at the manual would show way more features than I even thought about using, but by learning basic editing in emacs I’m already able to fully switch over to the Gnu editor.

This time I’m going to write about the cool stuff I’ve installed in emacs that you don’t see in the standard Sublime installation. Maybe Sublime has support for some of these functions, but not out of the box.

Magit

Magit allows you to control git from inside emacs. ‘But Matt,’ I hear you cry, ‘I can just leave emacs if I want to do git stuff! Why do I need this package?’ If I had to give you a a good reasons.. The whole thing is a pleasant user experience. Magit itself manifests as a new buffer splitting your window, listing the current status of the repository. Asking for a diff gives you the diff in another buffer; committing causes the commit message to open in a new buffer.. It’s very self-contained inside emacs. You don’t have to leave emacs for any part of the staging/committing/pushing, so you never have to break your flow. Even if you want to run an arbitrary git subcommand, Magit has you covered: The ! command while in the Magit buffer lets you specify subcommand you want.

But why alt-tab out to your terminal, then type ‘git commit -a’ only to be thrown back in to emacs for the commit message? Then you have to go back to the terminal to type ‘git push’.. What a chore. With Magit, I can just hit my Magit key binding (C-x C-a), then ‘c’ for commit, ‘-a’ for ‘add all unstaged files’, ‘c’ again, and I’m in a buffer where I can write my commit message. Write the message followed by C-c C-c to complete the commit. Then I just have to type ‘P’ twice to push, enter my SSH key-phrase, and that’s it.

Sure, Magit is definitely more of a convenience package than something that will supplement your coding skills, but it’s such a great convenience I can’t see why you wouldn’t want to use it!

Yasnippet

This one is a little unfair, because Sublime has support for snippets, and I just never really used them. Anyway: Yasnippet by João Távora allows you to use snippets when coding. Snippets are blocks of (usually) often used code that you can assign a shorthand command for. These commands take the form of a sequence of characters typed in to the editor followed by a tab. For instance, when writing Python I could type ‘cls’ followed by a tab and cls would be replaced by:

Yasnippet1

Emacs highlights ‘class’, ready for me to type the new class’ name. Once typed, I press tab again and I’m done. Snippets can contain more than one field to modify. In that case, tab moves you to the next field.

That happens to be one of the snippets you get when you first grab yasnippets via the package manager. It’s very simple to write your own, and dare I say, easier than writing the equivalent template in Sublime. The snippet above is defined like this:

Yasnippet2

If we wanted to save ourselves even more writing and not have to worry about writing the usual init function, we could modify it like so:

Yasnippet3

Notice how I’ve referenced $1 in two places and added a second field for us to potentially modify for when the new class is a subclass. By referencing a field more than once, the field is mirrored in other places. Expanding the snippet now gives this:

Yasnippet4

Notice how I’ve typed ‘Woah’ and it has automagically appeared in other places! Pretty cool. There’s a lot of potential in snippets to unlock, and they can save you from large amounts of typing. They’re a great feature to have, and I can’t wait to think up some useful snippets to improve my workflow.

Ace-jump-mode

One of the simplest packages on this list, ace-jump-mode by winterTTr is one of the most difficult to explain. Functionally, it’s a method of moving the cursor to the beginning of any word on the screen. The way you use it might seem weird at first, but it quickly becomes second nature. In this screenshot, my point is at the bottom of the screen but I want it to jump up to the creation of the Player object, because I just remembered I renamed that class.

AceJump1

Rather than move the point all that way (a whole 12 lines, so much effort), I can activate ace-jump-mode with my key-binding (Ctrl+C SPC), type the first letter of the word I want to jump to, and bam:

AceJump2

Ace-jump-mode now finds all instances of the chosen letter on the screen when it appears at the start of a word, and labels them with a letter of the alphabet. In this case, I want to jump to Player, so all I have to do is type ‘i’ and the point will jump to that location! I think it’s pretty neat in its simplicity, and saves some tedious point moving action you’d otherwise be doing.

That’s pretty much it! Before I finish this post, I’d like to take a little while to help out anyone having problems combining some of the packages discussed in these posts.

Jedi.el and multiple-cursors.el

These two packages get along quite well until you have multiple cursors active and decide to type a dot. Upon seeing a dot, Jedi.el decides to attempt to call auto-completion to help out with class methods and the like. However, multiple-cursors explicitly disables auto-complete mode because auto-complete doesn’t play well with it. The end result is that an error is thrown on typing dot, so only the first cursor actually types it. This is problematic, but I have a solution! In fact, I’ve submitted a pull request to the multiple-cursors.el repository that was recently merged. But for you people who don’t want to upgrade multiple-cursors to the latest pull or want to wait for a new release, here’s another solution that you can add to your .emacs after your normal multiple-cursors set-up:

(setq mc/unsupported-minor-modes '(auto-complete-mode flyspell-mode jedi-mode))

After adding this, you can type dots to your hearts content, even when using jedi.el! Enjoy.

Yasnippet and auto-complete.el

While not mentioned explicitly here or in the previous post, Jedi.el uses auto-complete.el for its auto-complete interface. Sometimes you’ll want snippets to appear in the auto-complete list. By default, auto-complete.el is coded to not include a yasnippet source. A source is just an abstraction that provides suggestions to auto-complete.el, with the actual method of finding said suggestions hidden inside the source itself. With a bit of tweaking, we can have snippets appear in all auto-complete suggestions regardless of mode by forcing the yasnippet source to be loaded in all modes. This is typically desirable behaviour, because if we have snippets for a mode, we usually want to use them! To do this, you need to add a hook to the auto-complete-mode in .emacs:

(defun ac-always-add-yasnippet-source ()
(add-to-list 'ac-sources 'ac-source-yasnippet))

(add-hook 'auto-complete-mode-hook 'ac-always-add-yasnippet-source)

Alternatively, you can add hooks on a per Major mode basis instead. For instance, you can add the same function as a hook to Python mode by replacing auto-complete-mode-hook by python-mode-hook.

That about wraps it up for this post. If I find a really cool emacs package that I have to shout about, I’ll be sure to mention it. Otherwise, next time we’ll be looking at Rogue Detective (working title), my really-kinda-lame-but-I-like-working-on-it Python roguelike developed in emacs!

Getting Virtual with Emacs (Part 2)

It’s been just over a month since I started properly developing in emacs, both at work and at home. I think it’s about time I wrote a little about my experiences.

First of all, while I’ve used vim daily at work, I don’t claim to be a vim power-user by any stretch of the imagination. Sure, I know how to navigate around quickly, and how to perform some basic editing, but I’m nowhere near the proficiency required to consider oneself a true user. For instance, I’m pretty terrible at VimGolf! When doing some heavy duty coding, I worked in Sublime Text 2. Even in Sublime, for me it was more about the syntax highlighting and pretty interface than it was about its cool features (with the exception of multiple selections). Like many beginner programmers, I haven’t yet took the time to learn my environment. Or any environment, for that matter! After watching some cool emacs videos, I decided I wanted to properly ‘learn’ emacs, and not just use it in the same way I use vim (in that case, why even try emacs?).

The question was: Could using emacs help my productivity? Sure, it wouldn’t help me think up or reason about code faster, but when an idea finally does pop in to my head, the last thing I want is for my editor to get in the way. In fact, an editor that could help me express my ideas faster than other editors would be a boon.

My first experience with emacs was with the in-editor tutorial. It teaches you how to move the point (cursor) and how to perform simple editing. The tutorial barely scratches the surface of emacs, but it does ease beginners into editing with emacs in a gentle, easy-to-follow fashion. After blowing through that and wondering what all the fuss was about, I would find the real power of emacs comes from its extensibility.

Emacs extensions

While emacs is a powerful editor in its own right, the huge amount of community support means that regardless of what language you’re using, someone has written an extension to make coding in it easier. If they haven’t (they have), rolling your own is just a case of applying your Lisp knowledge to the emacs equivalent, elisp. Don’t know Lisp? After spending enough time with emacs, you’ll at least have a good enough grounding in elisp to understand the odd bit of proper Lisp code, and enough knowledge of elisp to be writing your own little bits of code to enhance emacs. I realised that all the cool things in those videos I was watching were often showcasing packages (extensions) for emacs.

My first big programming project using emacs is a Python roguelike that I’ll talk about in future posts. Having written my previous Python project in Sublime Text, my first aim was to be able to do in emacs what I could do in Sublime. This turned out be simple, given how I’d barely given Sublime a proper workout in the first place. I wanted half-way clever auto-complete functionality, multiple selections, and a colour scheme that wouldn’t melt my eyes. After getting the basics out of the way, it would be time to explore the larger world of emacs.

As of emacs 24, emacs comes bundled with a package manager. It turns out there’s more than a couple of repositories for emacs packages, and it’s simple to add them to your emacs package list:

Packages

Once that’s sorted, you can bring up the package manager by using the ‘list-packages’ M-x command and browse the huge variety of packages available at your leisure:

Yes, 2048 has even invaded emacs

Yes, 2048 has even invaded emacs

Jedi.el – Python Auto-complete

In my Python setup, the Jedi.el package provides auto-completion. It’s a really clever piece of kit. Jedi.el will not only offer auto-complete suggestions based off the history of the file (giving auto-complete options for words already typed), but thanks to integration with a Python server called Jedi, Jedi.el will also have knowledge of your classes and function definitions. For instance, if you’re writing a method inside a class and type ‘self.':

Jedi.el Autocomplete

Jedi.el and Jedi will be able to figure out your class’s variables and functions and offer auto-complete suggestions. This is seriously useful. If you’re writing code and forget what arguments to pass to a function, you’re also in luck:

Jedi.el Function Signature

So yeah! Overall, Jedi.el is a really neat package. It has a few other features I haven’t tried out too much yet (‘goto definition’ for instance), but I’m looking forward to exploring this package a bit more. Installation was a bit more involved that most emacs packages, requiring some pip action to get the external Jedi server installed. But in fact, Jedi.el came out with a new version that makes installation simpler than before, so even that’s not really a valid criticism. Here’s how Jedi.el is set-up in my .emacs. In the latest version, the setup-keys variable isn’t required.

JediSetup

multiple-cursors.el – Multiple Selections

multiple-cursors.el is written by the wonderful Magnar Sveen of Emacs Rocks!. He’s produced a video about it that can explain it far better than I could hope to:

Installation is as easy as downloading the package via the package manager and inserting a few key-bindings in to your .emacs:

Multiple Cursors Setup

Sublime Text and Vim look-and-feel

I really like how Sublime Text’s default theme looks. I also love the simplicity of the Vim interface. Emacs, out-of-the-box, looks positively crowded compared to the very clean, empty Vim interface, as demonstrated by the last article:

EmacsTerminalFresh2To be fair, that picture is also from the terminal version of emacs, but the point stands. What on earth is that menu bar doing up there? And in the X terminal version, there’s a scroll bar and a tool bar? Vim doesn’t need these things, and neither do I! They simply had to go:

GettingRidOfTheChaffAnd what of my wonderful Sublime Text colours? It turns out, someone had already made an emacs theme with pretty much the exact same colours as in Sublime Text’s monokai theme! They imaginatively called it monokai. I installed it via the package manager, added

(load-theme 'monokai t)

to my .emacs, and whoosh!

MonokaiThat is one sexy looking emacs set-up. Yes, there’s a couple of mistakes in my .emacs, but I’ll survive. Another little thing enabled here is column numbering in the status bar.

Those of you with eagle eyes will spot little things in the .emacs like yasnippet, magit and ace-jump-mode. Those are some very, very cool add-ons. I’d like to write a bit about those next time. For those who are really impatient, both yasnippet and ace-jump-mode have been subjects of Emacs Rocks! videos.

Getting Virtual with Emacs (Part 1)

A Quick Introduction

We’re taking a short (okay, long) break from the Hullet Bells posts. It’s already been a couple of months since I last updated, mostly because progress has been slow. I keep having ideas on how to refactor the fledging script system, but never get around to actually implementing them. So that’s on hold for now. In the meantime, I’ve been messing about with a couple of other things. Today, I’d like to write about my experiences with Oracle VirtualBox and Arch Linux. This isn’t going to be a tutorial on installing Arch on VirtualBox by any means. If you want that, you’d do well to check out Josh Braun’s excellent guide. When the ArchLinux beginner’s guide was not enough, Josh’s advice helped grease the wheels.

Oracle VirtualBox (or simply VirtualBox) is a virtual machine (or VM). For those unfamiliar with VMs, the concept is simple: Rather than install a second operating system via the traditional fashion of re-partitioning your hard-drive and modifying the boot loader so you can dual-boot, you run your standard operating system and boot the new one as a program inside it. The underlying operating system is called the ‘host’, and second operating system is known as the ‘guest’.

I’m running Windows (the host OS) with Sublime Text and VirtualBox open. Inside VirtualBox is an Arch Linux instance (the guest OS), running Openbox, Emacs, and Supertux. Why would anyone do this? Well, there’s plenty of legitimate reasons. For instance, as a developer you might want to test software on multiple platforms without having to drudge through rebooting or using a different machine every time you want to try something new. Or maybe you have some old applications that only run in an older version of Windows, and you’d like to use those without dusting off your ancient Dell. These are fine reasons, and there are many more. My reason was a little stranger..

I wanted to learn Emacs

Why learn GNU Emacs? Well, why not? I’m already pretty familiar with Vim, although I’m by no means a wizard. After watching a few videos of people working with Emacs, I figured maybe it was time to give it a go. I would be one of the brave, the few, to see what life was like on both sides of the editor war.

‘But Matthew,’ I hear you cry. ‘You don’t need to install Linux just to use Emacs!’ And you would be completely right. While I was toying with the idea of installing Emacs for Windows, I remembered another thing I wanted to get to grips with: Linux. I’ve had several abortive attempts over the years, usually lasting anywhere between a week to a couple of months. The high entry cost of having to leave behind Windows and its myriad useful applications ended up being too much to bear. But the future is here, and now I can run Linux in my Windows, so I can Linux while I Windows. So I decided to give both of them a go. But first I’d need a Virtual Machine to run Linux on..

Virtually a Machine

After Googling for all of thirty seconds, I came the open-source VM VirtualBox. There is an alternative in VMware Player. I picked VirtualBox simply because I had to pick between the two. It turned out to be a great choice, with almost no problems encountered at all.

VirtualBox

What’s that in the preview box?

The interface couldn’t be simpler. You can run as many virtual machines as your system can handle, and creating a new machine is as easy as clicking the New button and following the wizard. All you need to set up at first is the machine name, type (Linux, Windows, other), and version (Ubuntu, XP, OS/2, etc). Allocate it some amount of memory, specify some parameters about its virtual hard drive and bam! You’re done! Grab an ISO of your favourite Linux distribution, pop it in the virtual drive, boot up the machine..

ArchInstall

And install it just like you normally would, with a few special additions/omissions for doing it in VirtualBox. Again, I’m not about to provide a big ol’ tutorial. That’s already been done, and it’s been done very well. Some of the nice features you get when working with a supported VirtualBox OS:

  • Shared clipboard!
  • Ability to seamlessly move the mouse and keyboard between operating systems without having to explicitly regain control from the guest!
  • Other stuff I haven’t yet discovered!

It’s super simple and a pleasure to work with, at least for my current requirements.

Arch Linux

Why Arch? Well, I’m a big fan of lightweight stuff. Arch comes with just enough to get you going, and a great package manager (pacman) to help you get the stuff that you want. If you’re bit of a Linux penguin, you may understand my surprise at the fact I had to download sudo! That’s right, that’s how lightweight it is. Installation and initial set-up took me about half a day in total. This is keeping in mind I’m not very familiar with Linux installation, especially after such a long period between efforts.

One of the things I really like about running a virtual machine is that you don’t really have to worry about irrevocably screwing up your physical box. Things like disk partitioning are usually terrifying exercises in double, triple-checking your numbers, especially on a system that already has an OS installed. So without those worries, I was able to play about a bit and step out of my normal comfort zone. After that, it’s just going through the motions of installing the guest OS, enabling the network, setting the clock..

Then it was time to install Emacs. I took a deep breath and typed

sudo pacman -Sy emacs

and it was done!

Emacs on the Terminal

I ran through the Emacs tutorial in about an hour. I was amazed at how different the whole experience was from Vim. The look and feel were completely different, for one thing. A menu bar? Wow, I hadn’t seen one of those on a terminal editor since Edit! And the editing wasn’t modal like Vim: None of this switching between Normal and Edit mode. Of course, there were also little surprises. Yank means ‘paste’ in Emacs, while it meant ‘copy’ in Vim. There’s zero one-key short-cuts for things: Want to go to the start of the line you’re on? Ctrl+A will do it in Emacs, while ‘0’ will do it in Vim. Want to copy that line? ‘yy’ will do that for you in Vim. But you’ll need to either ‘mark’ (select) the line in Emacs and ‘kill’ (copy) it without deletion, or delete the whole line and ‘yank’ it back in, with the end result of the line being on Emac’s equivalent of the clipboard. The key sequences for these look like:

#1: Ctrl+a (go to start of line), Ctrl+Space (Place marker), Ctrl+e (Go to end of line), Meta+w (‘Kill’ without deletion)
#2: Ctrl+Shift+Backspace (‘Kill’ the line and delete it), Ctrl+y (‘Yank’ the line back)

Woah. All the simple things suddenly seemed very difficult. In reality, you get used to the extra keystrokes quite quickly. And if it really bothers you, there’s Emacs extensions and modes out there for making Emacs behave more like Vim! Which brings up a good point: Emacs is extensible. Like, crazy extensible. Thanks to its built-in language Elisp (Emacs Lisp, a Lisp dialect), it wouldn’t be hard to implement a single key-chord command that would perform one of the above methods. Indeed, the same thing could be accomplished with a macro (a recorded set of key presses), a bit of functionality common to both Vim and Emacs. In any case, moving on..

Here’s how Emacs looks on the terminal:

EmacsTerminalFresh2

Not super hot. I spent a lot of time trying to get custom themes working and looking good, but it’s not an easy task. Especially when Emacs only wants to work with 256 colours! After a lot of wasted time trying to get a full colour terminal (if you figure this out, let me know), I decided to just install Xorg.

Emacs in Openbox

Initially, I went with the safe choice: The Gnome Desktop Environment. I’d used it in the past, and it was pretty okay back then. But wow, does it take a long time to start-up? And you have to log-in, what a mess. I ditched that quickly and continued my adventure outside my comfort zone. I installed Openbox, a lightweight window manager. When I first booted it up, I didn’t realise it was loaded:

OpenBox

It’s very minimalist.

That’s it. That’s what Openbox looks like. A right click reveals a menu in which you can open up X programs normally. I’ll talk about Openbox a bit later, but first, here’s what Emacs looks like under X:

EmacsOpenbox2

Ah, much better

Quite different, right? If the colour scheme looks familiar, it’s because it was inspired by Sublime Text‘s monokai theme. Installing the theme was easy. I had to configure the in-built (as of March 2013) Emacs package manager to use the Melpa and Marmalade package repositories, and then just downloaded the theme via that! A quick config change in my .emacs file, and there we go. The comfortable look of Sublime Text, the power of Emacs.

Next time, I’ll be posting about my experiences with emacs so far. I’ll highlight a couple of interesting features and extensions that I like, and maybe even convince some of you to give emacs a shot!

The Hullet Bells – Part 5 (Valuing your Input)

What is Input Handling, anyway?
Handling input is the process of taking the player’s key presses and converting them to actions in game. Pygame supports two methods for doing this:
1) Examining the state of the keyboard directly each frame, which I’ll refer to as the key state method.
2) Key events, where the game pumps an event queue that is populated only when keys are pressed and released.
Typically, we will attach a function to a key state or event and it will be called when the key is pressed or held.

For the key state approach, the part of your main loop that deals with input will look something like this:

...
  keystates = pygame.key.get_pressed()
  if keystates[pygame.K_UP]:
    moveUp()
  if keystates[pygame.K_DOWN]:
    moveDown()
... continues for each key tested ...

Meanwhile, if you take a key event approach, it will look more like this:

...
  events = pygame.event.get()
  for event in events:
    if event.type == pygame.KEYDOWN:
      if event.key == pygame.K_UP:
        startMovingUp()
      elif event.key == pygame.K_DOWN:
        ...
    elif event.type == pygame.KEYUP:
      if event.key == pygame.K_UP:
        stopMovingUp()
      elif event.key == pygame.K_DOWN:
        ...
...

The first obvious difference is that we won’t check the keys we’re watching for in the key event approach until someone actually presses a key to populate the events variable. The second is that we have a different idea about how movement should be handled: On key down, it starts and will likely continue every frame until key up. The key state approach typically moves players directly (I say typically, because we’ll see later on that it’s not always the case), while the key event approach modifies the player’s velocity, letting the game loop sort out actually moving the player. The former (direct movement) is frowned upon for a reason I’ll touch on later.

The Big Problem with Key Events

With that out of the way, I’d like to talk about one of the interesting challenges when implementing a multi-handler system. Suppose a player is moving their ship to the left when he pauses the game (switching the handler). If the player lets go of the left key and unpauses the game, what is their expectation? Is it:

  1. The ship comes to a standstill after unpausing, or…
  2. The ship continues moving to the left, with further movements to the left doubling the player’s speed, with right key presses causing the movement to stop while the key is held?

I have a pretty good idea myself about what I’d expect. The latter is a pretty serious bug, and it’s one I’ve encountered over and over again.

The above problems are caused when using key events for actions that require both the key down (begin movement) and key up (end movement) events to be caught in pairs. We have issues when beginning a movement, but then the corresponding end movement event is lost. In some implementations of key events, this can be achieved by losing application focus while the key is held, releasing the key, then giving the application focus again. The end result is that the game believes the key is still held down when it is not! Another example using our handler system: holding right and pausing the game will switch the current handler to the pause screen. Letting go of the key then sends the key up event to the pause screen handler, and not the game screen: The key up is lost. When we return to the game, we’ll see the dreaded behaviour above.

A brief aside to discuss the underlying method of moving a ship around the screen: Something you learn from a few game programming websites is that you should never directly modify a player’s position as a result of input. Instead, input should modify the player’s velocity. That way, because you’ve been a good programmer and decoupled the input and game logic, the game itself will be able to decide if the velocity applied will affect a movement. It shouldn’t if the player is at the bounds of the game area, for instance, but directly moving the player means you lose the information about how the player got out of bounds, so they may not be correctly placed back on the field. So directly modifying position is out. The idea that helps solve this: Key down means you affect the player’s velocity in some way, key up means you reverse that change.

This is a great piece of advice when implementing the movement stuff for your game. A common pattern I’ve seen on game tutorial sites is to add to the player’s velocity on key down events, and to subtract the same value on key up. This unfortunately causes the problems described above if a large number of precautions aren’t taken, including but not limited to:

  • Making sure a key up is ignored if we haven’t seen a key down from that key
  • Maintaining a list of all held keys when switching between handlers (can be rolled in to the above)
  • Making sure any paired events don’t let one half of the pair fire more than once (two key downs in a row, say)
  • When switching between handlers, checking every key in the new handler, seeing if it is waiting for an event, and firing it if the key state has changed (not ideal, because this means the player will have to repress the button after unpausing, say)

It’s a lot of messing about. I might add, it’s not wrong either (when you correctly code the safeguards). It can be a pain to code, and the edge cases extend beyond what I’ve outlined above. As a result, I attempt to keep things simple: we should never require an event pair in the first place. Instead, for movement functionality traditionally provided by events, I instead use the key state: A map of each key to a boolean value, either true for pressed or false for released. Every frame I check if the left or right key is pressed, I add to the velocity as normal. However, rather than subtract when the key is lifted, I reset the velocity at the end of the game logic for that frame. Suddenly, all of the above problems disappear: with no events to miss, we can’t get the velocity in a weird state, especially when it is reset at the end of every frame anyway.

This introduces something that some might be considered a bug: Suppose the game runs at 60 FPS and I’m moving to the right. That is to say, I am holding the right key. If I release the key after I’ve moved that frame and before the next frame hold the key again, it would be as though I never let go of the key in the first place. Given that we’re running at 60FPS, that gives you roughly 0.0167 seconds to push it. Most likely, you simply can’t release and re-press that fast. Even if you could, under most scenarios this is not a problem. Indeed, for movement this is the exact same behaviour you’d see with key events: On the second frame you would see a key up and a key down event, with the net result of no change in your velocity. There are conceivably scenarios where this is a problem. Consider you could charge a shot by holding space, and fire by releasing. If the player wants to fire a shot and then begin charging a new one, the key event approach would win out: Space bar comes up, space bar comes down, so you get both events on the queue and they’re processed on frame 2. Under the key state approach, we wouldn’t fire the shot. Again, this is a bug, but we would only see the problem if the player is really, really, really fast. I can’t stress how fast he would have to be! They would have to be faster than this dude. So we sacrifice a small bit of functionality in the case where the player is faster at pushing buttons than the world champion. It’s a sacrifice I’m willing to make.

In actual fact, key events will fall over here for movement too: If you tap the right key and get the key down/key up within 0.0167 seconds, the net result will be no movement, just like the key state approach. Maybe it’s not so bad after all! At least for movement. But what about performance? As it turns out, I did some number-crunching on my desktop computer and came up with the following: It takes one millionth of a second more to watch 10 key states than it does to wait for any number of key events (which takes no time, really, because the event loop isn’t populated until you actually press things). I think that’s acceptable! And even on a non-monster machine, it’s not going to take up a significant amount of time compared to the rendering or actual proper game logic. If you want to compare the approaches, here’s a gist of what I did. You’ll need pygame to run it, of course.

So, we’re trading a millionth of a second and a little bit of velocity resetting logic for the peace-of-mind of not having to deal with the edge cases of the key event approach for paired events. I’d call that a fair trade. Hullet Bells runs on a hybrid system: Key state-based function calls for movement (and later shooting), and key events for rarer events like using a special attack or pausing the game. In fact, the input handler presented in the gist is the same one used in Hullet Bells! Yes, it has some pretty gnarly named functions (what’s all this talk about callbacks? They’re just functions!), but it is basically the complete implementation of the system. One limitation is that I have no support for key combinations (say, calling a function if two keys are pressed, rather than just one), but for Hullet Bells (and many shooting games) this is an acceptable limitation.

The Hullet Bells InputHandler

The goal of the InputHandler was to allow isolation of the input logic between handlers, and to make programming input stuff as painless as possible. As long as you’ve defined the logic you want behind a key, actually attaching it is as easy as making the following calls:

# For key states
self.inputHandler.addPerFrameCallback(self._moveUp, pygame.K_UP)
# For key events
self.inputHandler.addPerFrameCallback(self._startMoveUp, pygame.K_UP, KEYDOWN)
self.inputHandler.addPerFrameCallback(self._stopMoveUp, pygame.K_UP, KEYUP)

These cause the event/key to be put in to lists that are evaluated every time inputHandler.update() is called:

  events = pygame.event.get()
  for event in events:
    if event.type == pygame.KEYDOWN:
      for keydownCallback in self.keydownCallbacks:
        if keydownCallback.key == event.key:
          keydownCallback.callback()
    elif event.type == pygame.KEYUP:
      for keyupCallback in self.keyupCallbacks:
        if keyupCallback.key == event.key:
          keyupCallback.callback()
...
  keystates = pygame.key.get_pressed()
  for perframeCallback in self.perframeCallbacks:
    if keystates[perframeCallback.key]:
      perframeCallback.callback()

For the key event stuff, it has just occurred to me that it would be nicer to store all KEYDOWN etc. events in a map of [event -> (key, callback)]. Also, for key states it might be nice to attempt to call all functions related to pressed keys regardless of whether a function exists or not, then we can eliminate the if statement too.. Anyway, those are actions for another time! As it stands, the input handler runs pretty well, so I’m not exactly looking to optimise yet. For the complete code, you can check out the Github or the above gist.

This was far too long, and didn’t really go anywhere. But that pretty much covers everything I wanted to talk about: Input handling in general, some of the problems you might have, and finally, my own implementation using the pygame library. I’d love to say the next post will be about the scripting system, but I’ll have to finish that before I can write about it. So if I want to write a post before then, I’ll talk about the game handler itself as it stands.

The Hullet Bells – Part 4 (Insert Title Here)

The title screen! At this point in game development, it’s probably a waste of time, especially if you’ve already decided to implement a handler system that will let you slot the title screen in at the end. But what’s done is done, so here it is!

The title screen!

They should have sent a poet.

Obviously a work of art. In this post, I’d like to talk a bit about the menu system and the Drawable class. The latter forms an important part of the game too, so it’s worth examining.

The menu system is fairly basic. The title screen handler has a collection of Buttons, with each Button containing a function that will be called when it is chosen. These buttons inherit from the TitleScreenElement class, which in turn inherits from the Drawable class.

class Drawable(object):
  def __init__(self, x, y):
    self.x = x
    self.y = y

  def setX(self, x):
    self.x = x

  def setY(self, y):
    self.y = y

...

class TitleScreenElement(drawable.Drawable):
  def __init__(self, x, y, text):
    super(TitleScreenHandler.TitleScreenElement, self).__init__(x, y)
    self.text = text
    font = pygame.font.Font(None, 24)
    self.renderText = font.render(text, 1, (255,255,255))
    self.textPos = self.renderText.get_rect(centerx=x, centery=y)

  def setFontSize(self, size):
    font = pygame.font.Font(None, size)
    self.renderText = font.render(self.text, 1, (255,255,255))
    self.textPos = self.renderText.get_rect(centerx=self.x, centery=self.y)

class Button(TitleScreenElement):
  def __init__(self, x, y, text, func):
    super(TitleScreenHandler.Button, self).__init__(x, y, text)
    self.text = text
    self.selected = False
    self.func = func

  def run(self):
    if self.func:
      self.func()
    else:
      print("No stuff!")

  def toggleSelect(self):
    self.selected = not self.selected
    if (self.selected):
      self.setFontSize(28)
    else:
      self.setFontSize(24)

The title itself is a TitleScreenElement, while the menu items are Buttons. As you may have noticed, each Button has a notion of being ‘selected’, which in turn affects how it appears on the screen. The title screen handler itself also contains a variable that tracks the index of the currently selected Button. It might be more accurate to call it the ‘hovered’ Button, as selection implies that it has been chosen. When a different menu item is chosen, the buttons toggle their selected status and change their font sizes.

Navigating the menu and activating a menu item is managed by an InputHandler (which we’ll discuss next time) and a number of functions. These functions adjust both the currently selected Button according to both the buttons and the title screen handler, as well as actually running the associated function. In the current version, one button will change the current handler to the Game handler, while the other terminates the game cleanly by ending the main loop.

... In the title screen handler ...
  self.buttons = [TitleScreenHandler.Button(self.game.xRes // 2, 300, "Start Game", self._startGame),
                  TitleScreenHandler.Button(self.game.xRes // 2, 400, "Quit", self._inputQuit)]
...
  def _inputQuit(self):
    self.running = False

  def _startGame(self):
    fadeToHandler(self.game.screen, 0.1, GameScreenHandler(self.game), self.game)

The logic for changing the selected button looks like this (without the corresponding _selectionDown function):

...
self.inputHandler.addEventCallback(self._selectionUp, pygame.K_UP, pygame.KEYDOWN)
...
def _selectionUp(self):
self.buttons[self.selected].toggleSelect()
self.selected -= 1
self.selected = (len(self.buttons) - 1) if (self.selected < 0) else self.selected self.buttons[self.selected].toggleSelect() print(self.selected) ... [/python] That covers the higher-level stuff for the title screen, so what about the Drawable base class we talked about earlier? Drawables are currently very simple: They contain an x and y coordinate, and nothing more. In the future, they will also contain Animations, the foundation for all displayed images. For Buttons, we currently create the text on the fly via the pygame.font library, but these will eventually be replaced by images if we're going to be making a seriously gorgeous title screen. Animations are only partially implemented, but will be heavily 'inspired' by the implementation in my platform game. An Animation is a set of one or more pygame surfaces, with the current image cycled every N frames. Animations may loop (not necessarily from the first frame). A Drawable will contain a collection of Animations, as well as an index to the current animation. It's a very simple system, and if we need to layer something more fancy on top of it later (particle effects, lighting..) it won't be too nightmarish. Next time we'll look at the input handling framework. The write-up is bit of a beast, but the actual logic is pretty simple! If you haven't already, you can see the list of all Hullet Bells articles here. There's even a link to the Github repo, if you can't wait for the play-by-play on this blog.