Dan's posterous

 

Native Instruments SCAM

You probably want to assign the 16 pads of the Maschine hardware to the 16 cells of the Remix Decks. Unfortunately, this will not be possible. The reason is that the depth of integration between the F1 hardware and Remix Deck exceeds what is possible through regular MIDI mapping in the Controller Manager.

COMPLETE and utter bullshit. They are doing this to sell more of their proprietary hardware and no other reason. While I don't own an F1 personally, I am told you will need to buy more than one if you want to control more than one Remix Deck at once - there is no way to swap or focus the active deck and there is no way to map other hardware to it. A reliable source tells me that a quick peek at the USB stream shows that they identify that an F1 is connected before letting the Remix Deck work with the controller to prevent people from connecting non-NI controllers to use the Remix Deck.

As a business model, I understand why they would want to focus on hardware rather than software (and essentially release Traktor for free with their controllers), but for users this sucks. Hard.

Posted May 1, 2012
 

Brief PIC24 assembly tutorial

For the past week I was trying to implement a simple program for a PIC24HJ128GP502 microcontroller in PIC assembly and found it hard to figure things out due to the lack of tutorials on 16bit PIC assembly (most sample code I found on Google were for the 8bit PIC16's), so I decided to write a brief tutorial so that other people won't have to figure everything out for themselves from the datasheet, like I did.

One thing I was struggling with is reading input pins, since the datasheet doesn't really explain much. For example, most PIC24's allow reading analog values from some of the pins, but if you want to use them as digital inputs (so they can be read from PORTA or LATA, for example), then you must turn off the analog to digital converter (which is enabled by default) for those pins. This tripped me up for quite a while, because RA0 and RA1 are also used for the analog AN0 and AN1. Reading those pins in PORTA would always result in 0.

Another thing that took a while before I figured it out is that the state of each pin is cached in LATx. That is, PORTA is the current state of the pins, while LATA is the cached state. This is important if you use the so-called read-modify-write instructions, such as those for setting a bit. To set a bit, the PIC reads the register, sets the bit and then writes it back, but when doing this with a register that isn't a physical register, but rather something that you can read from or write to that is mapped into the address space, the results can be unexpected. PORTA is one such register, since reading it reads the current state of the pins and writing to it sets the current state of the pins, rather than reading or writing a register. The state of such registers when reading may not be the same as when accessing it a second time (eg, to write it). Because of this, setting a bit may rewrite incorrect values for other bits. This can happen if a capacitor is attached to an output pin and that pin is just set to 1. Reading it now may actually return a 0, because the capacitor temporarily drains the pin, but the state of the pin is actually 1. If you were to now set a different bit on that port, the bit set instruction may actually write 0 back to the pin, instead of 1 as expected, driving the pin back to 0. To avoid this problem, we can instead use LATx. LATA stores the state that PORTA should be, even if the port is temporarily being drained.

For example, this code:

may result in PORTA being binary 10 instead of binary 11, if something is draining pin RA0. To fix this, we can use:

You can read more about this in section 9.10 of the I/O Ports manual.

Filed under  //   16bit   Assembly   I/O   Microcontroller   PIC   PIC24   programming  
Posted April 4, 2011
 

Game Engine

Lately, I've been working on a from-scratch game engine in C++. My main goals are ease of use (for the gameplay programmer), flexibility and performance.

To achieve these goals, I employed various tactics:

  • I achieve flexibility and extensibility by building the main data and logic around the concepts of a component entity system. This allows the games entities to be augmented with new behaviors simply by assigning new traits to them. The entities are, in effect, defined by their composition of traits. Since traits can be added independently, this allows for a powerful and flexible means of defining gameplay. Furthermore, as I will explain below, this organization scheme also lends itself to the performance goal by arranging data in a more cache-friendly manner than conventional object-oriented design does and by suitably partitioning data for parallel processing.
  • Different aspects of the engine communicate through an event system. Events can be posted by any module, may contain a variety of data, can be sent to specific receivers or broadcast to any that care to listen and are allocated from an extremely fast memory allocator. The event system allows modules to be independent, the only source of coupling being through the types of events they send and receive.
  • I achieve ease of use through a simple and convenient programming interface, built around the concepts of the entity system and event system. The interface is heavily augmented with template metaprogramming to remove any needless details, allowing the gameplay programmer to focus on the task at hand.
  • Performance is achieved through a design aimed squarely at multicore systems, use of memory pools with low latency O(1) allocation and deallocation, and heavy use of compile-time template metaprogramming to compile as many of the engine's high-level abstractions away into direct and efficient runtime code as possible.

I still have a lot of work to do before the engine can be used for real games, so we will have to wait and see how successful I was in achieving these goals. So far tests, benchmarks and profiling have shown positive results. More on this as the engine matures over the coming weeks.

Before I introduce the engines overall high-level architecture, I will briefly talk about the component entity system and its parts. I feel it is also important to define the terms I use when talking about my engine, since they diverge somewhat from those normally used when talking about component entity systems. I made these changes to the terms because I felt the common terms were too ambiguous and vague.

  • Entity: Everything in the game is an entity - the player, characters, obstacles, weapons, items, triggers, etc. Basically anything that can interact in some way, either physically in the game world or otherwise, is an entity. As far as the engine is concerned, an entity is nothing more than an identifier or key to which traits can be bound, though for the users convenience entities are normally encapsulated in a wrapper class containing convenience methods for dealing with that specific entity: adding, removing or retrieving traits, for example, amongst other entity management tasks. An entities functionality arises from its composition of attached traits. You can think of an entity as a duck-typed object built out of composition instead of inheritance.
  • Trait: A trait is simply a blob of data which may be attached to an entity. The existence of an attached trait  (or a combination of attached traits) defines the entities behavior and the data contained within a trait allows specific entities to be further configured or customized. The aggregate collection of traits defines the games dynamic data. In the context of component entity systems, traits are normally known as components.
  • Behavior: A behavior is an action or process which is applied to all entities which posses certain traits. For example, a behavior to move game objects around the world could be defined as taking a Position and Movement trait as inputs and generating a new Position trait as its output. This behavior would then be automatically applied to all entities which have both a Position and Movement trait attached. Because behaviors have this property (automatically being applied to all entities which can meet its input and output requirements), one can see how the simple act of attaching or detaching traits from an entity would change its functionality and characteristics. Int he context of component entity systems, behaviors are normally known as systems - however, unlike the systems described in most articles and Internet discussions about component entity systems, the behaviors in my engine are limited in what they may do: they can read their inputs, they can write to their output (a behavior may only have a single output Trait) and they may send events (either broadcast or sent to a specific destination). that's it. If it weren't for the event system, they would be pure-functional. This has two advantages: simplicity (ie, the code is easier to understand, verify and reason about) and parallelism (since each entity is independent and since inputs are read-only, it is almost trivial to apply behaviors to entities in parallel.
  • Event: An event is a small chunk of data (24 bytes, in my current implementation) which describes some action that either has been performed (notification event), or is to be performed (command event). Events contain a a type identifier, an optional mailbox (that is, the name of the event handler which should receive this event), a sender entity and between zero and three attributes. The attributes special types which can contain one of: a floating point number, an integer, an array of four bytes, a handle to a resource or a reference to a trait. Events are created throughout a frame in the game - by behaviors, or by external modules. Events are allocated from a special-purpose memory pool which is designed for extremely fast object destruction - essentially, at the end of a frame, when all events used during that frame are ready to be destroyed, a single atomic integer is reset to zero. That's it. All events gone! Allocation is O(1) as long as the block size (that is, how many objects are allocated by the pool at a time) is not exceeded. Once more than one block exists, and too many events are allocated to fit into the first block, there is a linked-list node traversal per exausted block (but since blocks typically hold hundreds or thousands of objects, its what I like to call "damn close to constant time complexity".
  • Event Handler: The event handlers are kind of like behaviors which are applied to groups of entities at once, instead of to single entities in isolation. Essentially, all events are stored in a queue until the handler is ready to process the event and then the handler processes the entire queue together as one task. Like behaviors, handlers have read-only access to the events traits (the traits are always the updated traits - that is, the traits as they are after the behaviors have been applied) and read-write access to a single, predefined output trait. Additionally, handlers can be passed the previous state of states, by passing them to the handler as event attributes (as traits passed as attributes are references to the read-only traits passed as inputs to the behavior which generated the event). An example of how an event handler may be used in a game would be to perform collision detection: all entities which have a Collidable and Position trait are processed by the collision behavior. This behavior creates a check_collision event, which contains the entities Position trait in one of its attributes. The collision_detection event handler receives a queue of all events sent during the previous update cycle - that is, all entities which take part in collision detection. The handler constructs a line segment which represents the distance and direction that the entity moved during the update cycle (the updated Position trait is the destination end point and the previous Position, which was passed in as an event attribute, is the source end point) and this line segment is checked against other entities' collision line segments to determine if any collisions occur ed. For every collision that occurred, the entities Movement trait is updated.

That's the engines entity system, which is the core of the engine and were the bulk of the logic lives.

Architecture

Now I will briefly talk about the engines overall architecture. You can get an idea of how the engine is constructed in the high-level architecture diagram above. You can see how the majority of the engine is composed from the entity system. The other important aspects are the input events, the renderer, the physics engine and (not shown in the diagram) the resource system.
Other modules and subsystems have not yet been developed, but when the engine is complete, it will contain a few more modules: audio output, an AI planner (for high-level decision making. Low level decisions can be made inside behaviors and handlers), a force-feedback generator, an embedded scripting language, perhaps a networking later (if I decide to give the engine networking capabilities), helper and convenience classes (polygons, planes, line segments, bounding boxes, vectors, matrices etc) and SIMD vectorized versions of these, where it makes sense. All traits are already aligned to 16-byte boundaries in anticipation of future SSE code.

So, allow me to talk about the other existing (and work-in-progress) modules:

  • Input System: The input system in a thread dedicated to input and rendering. That is because the underlying libraries I use require input and rendering to be done in the main thread. So, I move the entity system into a second thread and parallelize it further with a thread pool of worker threads. The input system simply polls for input periodically, translates it into events (based on the users key-bindings) and dispatches the events to the event system. The event handler then causes the input event to take its effect on the entity system.
  • Renderer: The renderer lives in the same dedicated thread as the input system. At the start of every frame, it gathers a set of rendering commands (passed to it by the handler which manages renderable/visual entities - the rendering commands are essentially a dedicated mini-event system) and turns them into commands for the GPU. Once rendering is complete, the front and back buffers are swapped and the new frame becomes visible. Finally, before the entire process repeats, the frame rate counter is updated. Note that rendering frame rate and simulation update cycle rate (that is, how quickly the entity system completes a behavior-event handler cycle) are independent and not synchronized. This means that the simulation could be updated multiple times between rendered frames (if the entity system is faster than the renderer) or that the rendering thread could be idle waiting for the simulation to update (if the renderer is faster than the entity system). Neither have drastic affects on the game (that I know of! This is something I will be testing in detail when the engine becomes more complete). The renderer is, as yet, mostly unimplemented, but will be OpenGL 3.x (with OpenGL 2 fallback) based. Though my initial games are 2D, I plan on implementing a deferred shading based renderer in the spirit of current 3D engines (with post processing if I can manage it).
  • Physics Engine: The physics engine is not yet integrated, but I plan on using the Bullet physics engine. The physics engine will be connected into the event system by implementing an adapter which converts physics callbacks to entity system events and converts physics entity system events to method calls into the physics engine. Behaviors and event handlers would need to be set up to keep the entity system in sync with the physics simulation maintained by Bullet. Keeping the physics isolated should also ease the task of providing the physics engine with a fixed update step time (physics engine simulations can become unstable if their simulation step fluctuates).
  • Resource System: The resource system lives in its own background thread, which it uses to stream resources (images, audio, map data, configuration, etc) from the disk in the background. When another engine module requires a resource, it notifies the resource system, which then begins streaming it into memory and provides the requesting module with a handle to the resource. To access the resources data, it must be requested from the resource system through the handle (locked for reading). If the resource has not yet been fully streamed, the caller is made aware of this and can choose to either be notified with an event, when the resource is available, or it can poll again later. Handles are also conveniently small chunks of data which can be passed around between modules and entities, as they themselves are small (4 bytes), yet they refer to data which may be extremely large. This is how, for example, the animation system works: the Animation traits have handles to various graphical resources required to display the animations. The handles are passed to the renderer, which requests the data from the resource system when it becomes time to render the animations. The resource system may also free stale data, to make room for fresh resources. Finally, the resource system is capable of compacting used memory by rearranging the data that the handles refer to, as long as no module has the handle locked (and if a module requests the data while it is being moved, as error similar to (or even the same as) when the data has not finished streaming to memory occurs).


That's pretty much it. More details will follow as the engine becomes more and more functional. I expect to write a few posts on the internal technical details, on the gameplay programming interface and on the game I will be writing using this engine.
But first, I will leave you with some technical details of the engine (language, libraries used, size, etc):

  • Written in C++, regularly tested in both GCC and MSVC
  • Developed on Windows 7, but am planning both an OS X and Linux port (and have made sure only to use portable libraries and code, so hopefully porting to these platforms won't be too dificult)
  • I use QtCreator as my IDE, but use Visual Studio for debugging (because I like the Visual Studio debugger better than gdb) and to compile builds for profiling (because I find PDB-based profilers better than gprof)
  • I've been using Intel VTune Amplifier XE 2011 for profiling (it can profile for general hotspots, concurrency, locks, memory bandwidth, memory accesses (cache misses, etc), branch misprediction and more)
  • I use the Simple DirectMedia Layer (SDL), version 1.3 (which is still in development, but already a lot more flexible than the old 1.2 release) for input handling, window management and to create background threads
  • I use Intel Threading Building Blocks to process performance critical code in parallel, for its concurrent containers (I use the queue and hash map containers), atomic integers, mutexes, timing and algorithmic templates. Basically, I let TBB create the threads for the code I parallelize for performance and SDL to create the threads used for asynchronous processing.
  • I use OpenGL 3.2 core profile (with an OpenGL 2 fallback) for rendering
  • I use some of the Boost libraries where they make sense. Currently I only use a very minimal amount of Boost, but I may use it more in the future
  • The current codebase achieves almost 98% CPU utilization on my dual core test machine. Sadly, I expect this to drop as I introduce more inherently-sequential code into the engine. I will be testing it on a quad core Core 2 and a 6-core AMD Phenom II in the coming weeks though. With a bit of luck, it will scale reasonably well to the higher number of processor cores
  • The current codebase is only approx. 3500 lines of C++, including comments. About 50% of this is template classes (slows down compilation, but the results - mainly a cleaner API, but it also allows some code to be inlined and better optimized - are worth it, IMHO)
  • For the embedded scripting language, I am currently considering Falcon, Squirrel and Lua (possibly with LuaJIT). I have not yet decided when I will add scripting, what I will expose to the scripting language (the entity and event systems, I imagine), or which language I will embed
  • I am keeping the engine closed-source for the moment, but I am considering the possibility of open sourcing it in the future. In any case, there are some components I intend on factoring into their own libraries (some memory management code, at the least) and releasing as open source, though I don't yet know when I'll get around to doing so

Before I go, shameless plug:

  • The game for which this engine is being built is going to be AWESOME, but you're going to have to wait until I have something to demonstrate before I give away any details. Don't want to ruin the hype before it even started  ;-)
  • So far, I have written the engine by myself, but I have assembled another programmer, a professional game tester/designer/level-designer and an artist, so with a bit of luck, this project is something we can complete within a reasonable time frame

Hopefully somebody found this interesting.

Filed under  //   C++   Threading Building Blocks   concurrency   entity system   game development   game engine   gamedev   multi-core   programming   software architecture  
Posted January 29, 2011
 

NetPad/Nexus, an old executable

Click here to download:
Nexus.rar (880 KB)

In a previous post, I wrote about a project I once worked on: Nexus (formerly NetPad).

Well, I just came accross a Windows executable of an old and early version! I've uploaded it here, in case anybody wants to try it out.

Sadly, the code seems to be lost forever :-(

Posted September 6, 2010
 

Things I'm working on now

So, since I've documented some of the things I've worked on in the past, I figured I may as well tell the world what I'm working on now. So, here's a list:

  • My startup. Basically, a domain specific spreadsheet. Expect to hear a lot lot more about this as I get closer to launching. For now, I'll continue working on it quietly.
  • The ANI programming language. An upcoming dataflow language designed for ease of use, performance and easy/safe parallel programming. I am currently working on, ever so slowly, the code generator (instruction selection, register allocation and so on).
  • Clojure Games, a website dedicated to game programming in the Clojure programming language. I am slowly working on content (articles and sample games) for the site.
  • A number of web development projects, in Python and Django. Basically, stuff to pay the bills.
  • I am heavily involved in the Python Ireland community and am currently working on a little project to promote Python to Irish universities as a suitable language to teach programming to beginners.

That's pretty much everything I'm actively (or semi-actively) working on, though there are a number of projects which I was working on or involved in, but am idle at the moment:
  • The Renraku operating system.
  • The Musca window manager community.

Filed under  //   programming   projects   technology  
Posted August 5, 2010
 

Things I've worked on #4

In the interest of immortalising prehistoric projects I once worked on, I decided to post them to the wonderful world of Internet.

This is by far the projects I put the most work into, but also, sadly, the ones I can talk about the least. Such is life in the world of professional software development.

I was at Cellusys for a six month internship while I was still studying, and then, a year later, I went back to work there full time for a year and 9 months. In that time I worked on a number of large projects and learned a lot of buzzwords: Java, Spring, JNI, SS7, GSM, SIGTRAN, Diameter, SMPP, ASN.1, MTP3, M3UA, SCCP, TCAP, MAP, SM-TP, SCTP, SMTP, Terracotta, Hazelcast and probably a bunch of protocols and technologies I've since forgotten about. I was involved in all aspects of the various projects, including planning, research, design, development, testing, installation and support. I attended the GSMA Mobile World Congress in Barcelona on behalf of the company and travelled to the middle-east to perform an on-site installation. Overall, a very well rounded experience.

I primarily worked on three projects while at Cellusys:

1. The SMS Defence anti fraud and anti spam platform. By extension, I also worked on the flagship CSS7 platform, on which most of the products are built, and related components (SMS message codec, SNMP notification system, application logic for blacklisting, whitelisting, throttling messages, work distribution and much much more). This was a large clustered system which, to put it simply and briefly, allows mobile network operators to monitor and filter SMS messages on their network in order to prevent fraud and spam and essentially protect themselves and their customers, especially from messages originating outside of their networks.

2. Prepaid SMS billing system. This system was designed for a niche use case where certain mobile network operators are not able to provide SMS messaging services to their prepaid customers while roaming on foreign networks, due to lack of billing infrastructure. This system basically processes messages sent by the networks roaming customers and performs billing operations, if required. I was responsible for the billing aspect of this project, which consisted of communicating with the networks billing servers via the DIAMETER protocol. A lot of time and effort was spent into insuring that there is absolutely no room for failure in the billing process, that every transaction is fully audited and that all billing operations are performed correctly.

3. Parental Control system. This is the only user-facing system which I worked on. It basically provides a web interface (two actually, one for end-users and one for the networks support team) which the users could use to place blocks on their children's phones. It allows parents to blacklist and white list specific numbers, blackout messaging during certain times of certain days (eg, week days during school hours) and limit how many messages the children are allowed to send in a given time period (daily, weekly, monthly).

Besides the above projects, I was involved in R&D into various technologies and components for use in the company or future products.
I could probably talk about these projects forever! They were quite large and consisted of a lot of different aspects and components - especially the clustering/distributed aspects, since this is an area of personal interests. But, of course, since most of that stuff is proprietary technology, I don't think there's too much more I can say, really, above what I've already said here.

Filed under  //   programming   projects   technology   things_ive_worked_on   work  
Posted August 5, 2010
 

Things I've worked on #3

In the interest of immortalising prehistoric projects I once worked on, I decided to post them to the wonderful world of Internet.

This post is a collection of less notable projects which I've worked on over the years. They're not interesting enough to talk about on their own, but perhaps as a collection, they will be of interest to somebody.

- - -

The most interesting of these projects was a map creator tool for Google maps, on which I worked on the back end PHP code. It was never released publicly, since it was contract work for somebody, but since it was a long time ago now and since Google now has a much more powerful map builder, I guess its OK to talk about it now.
What makes this project interesting is that it was completed before Google released the official map builder tools! This tool allowed you to create your own custom maps and place markers on them. Each marker could be given a a title, description and a photo. I even used the demo deployment a few times to give people directions. Of course, it has since been completely overshadowed by Google's own version. A version of the code is available for you to look at here.

- - -

I once worked on a project with my cousin, which we sadly never completed. It had two components - a desktop client and a web based version. The best way to describe it is to say that the web based version was basically like Etherpad - but a few years before Etherpad existed. If we had known, perhaps we would have completed it?
Originally, I named it Netpad, since it was a networked notepad.exe clone. Later we renamed it to Nexus.
The idea was that it would enable remote pair programming. You basically had a programming text editor (with all the usual features - syntax highlighting and auto completion and so on) with two editor windows side by side. The left-hand one was your local editor, which you could use to edit whatever file you wanted. The one on the right was the remote editor and it displayed whatever the other person was working on. Of course, you could scroll through the other persons code (you didn't have to simply watch what they did) and even type into it.
Together with a chat system, the idea was that you could work on your own code, while seeing what your partner worked on. Then if either of you needed help, you could type into each others editors. Or highlight code or whatever you wanted.
We had a number of different prototype version (I believe the latest version of Nexus also had a server with basic source control), but we never got far enough to actually release anything. A pity really.

- - -

Originally, I got into programming for game development, so its only natural that I tinkered with game programming over the years. I've worked on countless prototypes and concepts, but very few complete games. In fact, I only ever fully completed three, so I'll briefly talk about those here.

The first one was a Tetris game. I was in school at the time and I remember early one morning (or, late one night, depending on your perspective) I realised a very simple way of rotating blocks by flipping a 2D array. I think it was about 5 or 6am. I wrote it up real quick and then the next day when I got up, I wrote a tetris game. It had a title screen, pause menu, high scores table, sound effects and background music as well as some pretty MS Paint graphics. It was programmed in C++ using the Simple DirectMedia Layer for input, graphics and sound.

The second game I completed was a ball bouncing game. It started off as a game of Pong, except that I removed the second paddle and instead the goal was to bounce the ball as long as you could. The paddle was at the bottom of the screen and gravity pulled the ball down. If you missed the ball, the score multiplier reset to 1 and the balls speed reset to its initial speed. When, instead, you bounced the ball, you received points equal to the current multiplier, the multiplier was incremented (to a maximum of nine) and the balls speed was increased. You had about two minutes to get as high a score as you could.
The game had particle effects (every time the ball bounced off the top of the screen, an explosion of particles was generated), floaty numbers were sent flying from the paddle every time you bounced the ball, sound effects, background music, a high score table, pause menu, main menu and a selection of either keyboard or mouse controls. It was programmed in C++, using the Simple DirectMedia Layer for input, graphics and sound.
The story behind the game is kind of funny too - I wrote it the day before my first year university exams, to annoy a friend, who was annoyed at me because I wouldn't study for the programming exam that we had the next day. I think the game took about four or five hours to complete (though I did tweak it over the following weeks). The game was quite popular amongst my university friends for a short time too.

The third, and final, full game I completed was a very simple dungeon crawler. It took about a day to write and is the only one of the three games that I know I still have the code for (and may upload some day). Again, it was written in C++ and used the Simple DirectMedia Layer. This one didn't have any sound or menu. Its a tile based overhead scrolling game and the goal was to navigate a small map, kill monsters and collect gold form treasure chests. I guess calling it a complete game is a bit far fetched really, since it only had a single, small level and no health power-ups (which made it quite hard to play!), but I originally wrote it because I was going to write a tutorial series on game programming. The code is fairly easy to understand and heavily commented.

- - -

Once upon a time, in second year of my Computer Science degree, for probability class, we were to write a program which generated graphs for a number of probability formulas. Well, the task was so boring that instead of doing what was asked for, I wrote a complete graphing calculator instead and provided a little menu for "preset" formulas to graph (the ones required for class).
This tool was written in C++ using the FLTK GUI toolkit, OpenGL to render the graphs and the Squirrel Scripting language to write the formulae. The graph could be panned and supported zooming. It was even able to draw circles (by letting you set two outputs for the formula, instead of one - basically f(x)=y and f(t)=x,y) and you could even make the graph change colour (by setting r, g and b output variables). You could select between smoothed line graph, bar charts and filled line graph. Needless to say, it did a lot lot more than we were required and blew some of my fellow students away :-D For the most part, it took about a day to program.

Filed under  //   game development   programming   projects   technology   things_ive_worked_on   web development  
Posted August 5, 2010
 

Things I've worked on #2

In the interest of immortalising prehistoric projects I once worked on, I decided to post them to the wonderful world of Internet.

I worked on a power profiling project for audio-event detection with Graham Healy as my third-year Computer Science project in Dublin City University. The project was supervised by Prof. Alan Smeaton.

The Dublin City University Center for Digital Video Processing was working on wireless sensor network technology in the area of audio event detection. For this project, we worked together with one of the CDVP PHD students, who was actually implementing the audio event detection system. Our project consisted of taking his research, porting it to actually run on the wireless sensor network motes (so, to nesC and TinyOS) and then to power profile it and determine if there are any techniques or insights which the CDVP could use to retain the highest event detection accuracy with the lowest power consumption.

Our work consisted of porting MATLAB code to TinyOS, simulating the event detection procedure in the Avrora TinyOS simulator and generating a power profile for the task. We then analysed both the accuracy of the event detection techniques and the power profile to find a balance between power consumption and accuracy. To do this, we built a tool in Python to visualize the audio data and event detection, so that we could determine accuracy and correlate it to the power consumption. We then repeated this for a number of different variables (event detection techniques, audio window sizes, sample rates and so on) until we found the "sweet spot" where power consumption was as low as possible without having a detrimental effect on the quality of the event detection.

For this project, we used a mixture of nesC (the event detection code), Java (the Avrora simulator had to be extended by us) and Python (the analysis tools which we wrote for this project).

Our approach, analysis and results is contained in the attached PDF files. The functional specification is NOT missing a page, even though the numbering seems to suggest that it is.

Click here to download:
presentation.pdf (171 KB)
(download)

Click here to download:
Functional_Specification.pdf (681 KB)
(download)

Click here to download:
Technical Document.pdf (622 KB)
(download)

Filed under  //   hardware   programming   projects   technology   things_ive_worked_on   wireless sensor networks  
Posted August 5, 2010
 

Things I've worked on #1

In the interest of immortalising prehistoric projects I once worked on, I decided to post them to the wonderful world of Internet.

I'll start with one of the more interesting projects: A Framework for Augmented Reality Applications.

I worked on this project with Graham Healy as my Computer Science final year project in Dublin City University. The project was supervised by Prof. Alan Smeaton. The concept was to build an augmented reality "platform" which allows one to quickly and easily build AR applications. Since we didn't have that much time, we specifically limited it to 3D audio-based feedback (that is, no visual feedback at all, though our extensible design would have made it trivial to add visual components later). In short, we designed and built a kit which tracked you're movement, location and orientation and provided you with localised 3D audio feedback.

Some of the demonstration applications which we worked on were:

A navigation application, for example, to help blind people navigate. You defined a set of waypoints and it would guide you on a path through them until you reach the destination. It did this by continuously playing a sound at the location of the waypoint (that is, the 3D audio made it seem like it was coming from that location, though in reality we generated it dynamically and played it with a pair of wireless headphones). Since we had sensors to detect your position and orientation (amongst other things), we could detect when the user walks into the waypoint, at which point the sound would stop playing and the next one would start. The attached technical specification has some more details on this application.

Another demo application we worked on was to simulate a band playing music. A number of synchronised virtual musical instruments were scattered around the room, allowing the user to walk around and between them, as if they were actually amongst the musicians.

The final demo application I will mention is one where we detect walls using an ultrasonic sensor. When the sensor detects a wall, it repeatedly played tone. The speed of which the tone was repeated was dependent on the distance to the wall. This meant that if you shook your head form side to side, scanning the length of the wall, the sound played effectively tapered off into the distance giving you a recognisable audio representation of the shape of the wall.

The kit consisted of a number of hardware and software components. The headset consisted of a pair of wireless headphones, a Ubisense location tracking tag, a digital compass, an accelerometer, an ultrasonic sensor, a PIC microcotnroller (which basically multiplexed the signals from the sensors) and an RF-transceiver (we actually used an XBee Zigbee wireless sensor network mote for this for convenience, since we had them anyway, though it would obviously be cheaper to use a normal RF-transceiver). The software consisted of a number of server applications which communicated over TCP/IP. We had a number of these to preprocess the signals from the sensors (eg, to apply calibration data), one to generate the 3D audio (we had to run this on a computer of its own because we needed a specific sound card with HRTF support, to improve the quality of the 3D sound) and an application which bound everything together to actually create the user experience desired. The software was written in a mixture of PIC assembly, C, C++ and Python.
A full set of technical details are provided int he attached PDF documents, so be sure to check them out, if interested.

See the attached files for more details. I have included a few photos and the specification. Sadly, the final versions of the specs seem to be lost to time, so please excuse the typographical errors and incomplete appendices in this version.

One thing you will notice from the photos is that the final version was mounted on a cardboard box, suspended a few inches away from the headphones. This wasn't merely to improve the fashionable aesthetics of the head unit, but in fact to combat the effect of magnetism (from the headphones) on the digital compass. A non-prototype implementation would obviously properly shield the components to avoid this, but hey, we built it all (including the cost of the sound card) for about €500.

A side note: Graham went on to do a PHD in Dublin City University and spent a short time improving on our original design. The hardware has since been redesigned and professionally manufactured by Tyndall Institute using high-grade components. The accuracy has increased by a few orders of magnitude, but so has the cost!

 

(download)

Click here to download:
Functional Specification.pdf (402 KB)
(download)

Click here to download:
Technical Specification.pdf (1.67 MB)
(download)

Filed under  //   augmented reality   hardware   programming   projects   technology   things_ive_worked_on  
Posted August 5, 2010
 

Component Software

 

Advance apologies for bad formatting - seems posterous isn't the greatest for formatted text!

Clojure makes a lot of things really easy. Just for fun, I implemented a simple system for constructing a component based software architecture where each component runs completely independently and asynchronously. Components communicate by sending events to each other, achieving a flat software hierarchy.


We can test this by creating and running some test components:

Comp1 contains three event handlers - :init, :print and :set-msg. Comp2 only contains two event handlers - :init and :print.
:init is called to setup the components state and trigger any initial events. :print will print the contents of the states :msg slot to the terminal and :set-msg will change the :msg slot to the value of the events message.

Running this sample will output the following:

 Ho
 Hello

It becomes more interesting if the components do a little more, for example:

Running this will output:

 Hi
 Hello
 Ho
 Hello

Note how the first event gets processed before the secod one - ie, Comp1's :msg has not yet been modified.

Of course, you can set up all sorts of crazy chains of events with many event types, complex logic and so on.

What would be really nice is a DSL designed for writing these component-based programs, since calling run-components and make-components manually is a little messy. Something like the following would be nice:

Luckily, this is the kind of thing Lisp is good at and Clojure, being a Lisp dialect, has everything we need to make it happen:

These macros transform our convenient little DSL into a set of calls to run-components and make-components. Success! We can now write fun little component-based programs, arranged in a flat hierarchy, using an anonymous (sender never knows who receives the events; receiver never knows where the events come from) event-based messaging system.

 

Filed under  //   clojure   concurrency   programming  
Posted December 1, 2009