The Memory Project

memory from tekt.

Simulation system

The memory project is written in C++, using openFrameworks.

The system is a simulation of a set of entities, interacting and following behavioral rules and physical forces.

Memory’s world is defined as a cube of space. Every living entity in the simulation exists at some position within that space. Entities experience forces applied to them based on their proximity to certain other entities and the connections between entities.

There are several types of entities:

  • Observers are the main entities in the simulation. They represent people, who can experience events (occurrences). Every observer has a limited lifetime, and when it dies, its connections to other entities are severed, which in turn affects the behavior of those entities. Observers also maintain connections to other observers that shared common experiences with them.
  • Occurrences are the secondary entities in the simulation. An occurrence represents some event that occurs within the world. Events are created at randomized locations in the space over time. When an occurrence is created, it has a certain range. Any observer within that range becomes connected to that occurrence (in other words, they have a memory of the event). However, if no one is in range when the occurrence is created, it disappears as if it never happened (“if a tree falls in the forest…”). As observers die, their connections to occurrences are severed, and when there are no observers left that remember an occurrence, the occurrence is lost.

Physics system

The movement of the entities is controlled by a basic particle system. The initial version is hand written in C++, though later versions may try to offload some of that work to the GPU. Every entity has a position and a velocity. On every update cycle, a set of behaviors (rules) applies forces to the entities which are applied to the velocity which is in turn applied to the position. There are several types of behaviors:

  • Bounds – causes entities to bounce off the walls of the space
  • Noise force fields – uses noise patterns to push entities in different directions to add some variation
  • Drag/damping forces – causes entities to slow down unless force is continually applied
  • Anchor point attraction – pushes or pulls entities to their original location when they were spawned
  • Connected entity attraction – pushes or pulls entities based on the set other entities that they’re connected to.

Rendering system

The rendering system uses standard openFrameworks OpenGL. It is grouped into a set of renderers which each draw some type of element in the scene:

  • Observer renderer – draws markers at location of each living observer entity. It varies their opacity based on how much of their lifespan remains.
  • Occurrence renderer – draws markers for occurrence entities, varying their opacity based on the collective remaining life of the observers connected to it.
  • Connection renderers – draw lines between connected entities (observer/occurrence and observer/observer). The colors of the ends of each line are based on the entity at that end of the connection, which produces a linear gradient between the two sides of each connection.

A set of additional controllers manage things like the camera and applying GLSL shader-based post processing.

Eventually, the entity marker rendering will be done using GPU instancing to improve performance. Currently, roughly 71% of the time in each cycle is rendering vs 27% simulation updating.

Simulation events

The simulation provides a set of events which are triggered when various things happen, including:

  • Observer spawned
  • Observer died
  • Occurrence spawned
  • Occurrence spawn failed (no observers in range)
  • Occurrence died

Various components in the system listen for these events and perform various actions in response to them, such as logging, spawning animation objects, or sending messages to external systems (via MIDI or eventually OSC).

Parameters and configuration

Each controller (such as a renderer, behavior, or larger subsystem) is controlled by a set of parameters, where each parameter is either a simple value (boolean, number, vector, color, etc), or a sub-grouping of other parameters. It is implemented using a set of extensions to the standard openFrameworks ofParameter classes. The extensions provide additional functionality such as default values and JSON I/O, as well adapting their behavior to my own sense of coding taste (nice C++11 lambdas, etc).

Control system

The parameters are bound to GUI controls using the ofxGui and ofxGuiExtended addons. The GUI isn’t great, but it’s usable.

The system also supports configurable MIDI input/output parameter bindings. OSC support is in progress.

Upcoming features

  • Navigation entity system, which creates markers which trace along the network of connected entities.
  • OSC parameter mapping.
  • OSC event output, with full event data, as opposed to the midi implementation which just sends out a note to indicate when an event occurs.
  • Instancing-based rendering.
  • Possibly use OpenCL for behavior calculations (because GPU?).

Why openFrameworks?

These days most of my work tends to be in TouchDesigner, so naturally that was my first instinct. While TD is awesome for a lot of things, it didn’t seem suited to the type of behavioral simulation at the core of Memory. It would definitely have made the rendering much nice and more efficient, but implementing the simulation logic would probably have involved a large pile of Python awkwardly shoved into TD’s declarative-ish frame-based evaluation. Given that, I opted to do it using one of the more standard creative coding frameworks. I started with a prototype in Processing (t3kt/memory-prototype). Eventually I realized that it wouldn’t work well both due to the performance and the relatively small set of easily usable libraries (as compared to oF). I had used openFrameworks for a few previous projects, including Bleepout, so I went with that instead.

There are some problems with using openFrameworks. For example, I explored a bunch of the available oF GUI addons each one of them had some limitations that I didn’t like, but ofxGuiExtended was less problematic than the others. It’s still somewhat unstable and doesn’t support things like scrolling to access controls when the panel is taller than the window. This is somewhat indicative of how the openFrameworks addon ecosystem tends to work. There are lots of libraries available, but most of them aren’t really fully fleshed out. Also, there is a real lack of documentation throughout the core library, and even less in the majority of the libraries that I have encountered. There is a decent amount of higher level tutorial/guide-style documentation available though, which is really useful (such as ofBook).

I have some coding stylistic issues with oF and its community of addons (naming, global namespace imports in headers, etc). The library also encourages the use of global functions which make it simpler for beginners but also mean that there are lots of extraneous bits of code running frequently. That said, I can get over that, and in my own code I can be more picky about that stuff. Also I can redirect some of that obsessiveness back to the community (see elliotwoods/ofxLiquidEvent/pull/2).

etc

Memory’s source code is available on github: https://github.com/t3kt/memory

The project uses several oF addons: