During the AiGameDev Conference 2010 at Paris Claudio Pedica, a researcher at CADIA Labs, Reykjavik University, currently collaborating with CCP Games presented us their new social interaction system in a talk titled Human Territoriality for EVE Online’s Socially Smart Characters.

This was a short preview on how the space stations are going to be populated in the next EVE online expansion.

The main focus of the research conducted by CCP is based on the lack of feedback in games when it comes to interactions. For example if you pass besides a game character, he doesn’t notice you or in worse cases, if you bump into him, the most common reaction is that he won’t even step aside and your character stops dumbly like when facing a wall! Fortunately, not all games react like this. In GTA when you hit someone or pass close to him he steps aside or even says something to you.

What the guys at CCP are trying to do is to recreate an interaction system based on how humans practically cope with interactions and how they react to other people’s physical presence.

In the demos shown we could see the character move close to a group of NPCs; when close enough, the group understands that this character wants to be part of their group so they take a step aside to form a bigger but still homogeneous group.

The “circle” itself is, in this system, the interaction unit; each character and group possesses one and each unit is designed for a particular type of interaction.

1) First there is the personal circle which defines the limit of the body area.

2) The external circle is used for the close social interactions like dialogs or people walking near (also true for groups).

3) An inner circle is also defined by the participants of the group: a circle defines its body and the external circle of it. In another example we saw one participant moving a few steps from his former position and the rest of the group following him to recreate the circle.

I actually don’t play EVE Online at all and I don’t really know what they are planning to do with the new space stations system but having this feature is über-cool and I think it would really improve the experience of the players (at least contributing to a more immersive experience).

Having people walking from one spot to another or joining conversations or even reacting to the players’ presence will surely make us feel more in a real living world. When you think that some games only feature static NPCs, this talk gets even more impressing!

Human Territoriality for EVE Online’s Socially Smart Characters by Claudio Pedica can be downloaded from the AiGameDev website.

Our previous experience with localization has been pretty painful. Everybody can remember those German texts that were at least twice the control size. From our experience, we thought a tool could be use to help managing and creating localization. Especially when the task is outsourced to some studio living in another time zone. The goal is to provide an interface between the designer and the “localizators”. The tool will contain information about text extent and will provide a preview of the text inside its control.

The feature list will be announced in the following weeks. But here is a preview :
Qt has been chosen as Mojito, our proprietary engine, runs on the 3 main OSes. The file format will be XML, but the tool architecture will ease the implementation of other format as well. Platform specific text can be inserted ( such as specific hardware name ). Support for exotic language such as Chinese and Arabian is planned.

As the tool matures, we will discuss design choices we made on this blog. Developers in charge of a specific feature will also have the opportunity to expose their work.

Stay tuned, the first code release is approaching.

This is why we decided to automate our “Game Content Pipeline” with a continuous integration process. To do this, we used “Hudson” simply because it wasn’t too difficult to implement within the Mojito Engine and secondly because the solution is quite complete.

One of the first step I had to undertake within this pipeline was to import all assets from a graphical software and export them to a format our engine will be able to use. Collada is perfect for this job and this is why we went on to create a Collada importer and a viewer which runs in the Mojito Engine.

For the first time we are proud to present you, the star of all stars, “Mr. Duck” from Mojito’s viewer.

It’s going pretty well, faster than I originally expected. I started by translating the display part, initially running with DirectX on the Windows version and now running with OpenGL. The result was as expected, and that’s how we were able to post the first screenshot you’ve seen in an earlier post. We rapidly figured out that the input system provided by GLUT was not exactly what we needed. So we decided to rewrite that specific engine part with SDL in order to have not only the keyboard but the joypad too!

And I’m proud to say that everything is now fully working My next task will be to port the sound system of the engine.

As you can see in World of Goo, created by 2D Boy, they had about 17% of their sales on the Linux platform. It looks like a lot of open source gamers are just waiting for some stuff to play on the penguin!

So, be ready, Linux-users! Because in a not so distant future, you will be able to run on your hardware some Fishing Cactus games and show off in front of all your friends!

Luckily, they can discover the joy of working with the open-source “OS for the poor”, Linux!

We have already improved the most current platforms on the market with our kickass Mojito Engine, and we are now currently working on the Linux version.

And just because we like you guys, readers of this blog, we will offer you the exclusivity of the year : the first screenshot of a not-so-mysterious project, running on Ubuntu!

I know how much it’s beautiful, but don’t cry yet, keep your tears when it will be fully functional so stay tuned!

Compilers only allow TLS on POD ( Plain Old Data ) types, no object can be used. This extension allows to use all types of objects, as long as it has a default constructor.

Allowing complex types in TLS leads to 2 challenges : allocating enough memory to contain the object and calling constructors and destructors on thread startup and exit.

In the previous version, the memory used for TLS was a table of 4 bytes-long items. We could have used this system to store a pointer to the complex object, but we tried to avoid dereferences as much as possible. The system is then changed to contain any size needed. Each TLS instance contains the index inside a buffer. This buffer is created for each new thread and contains the memory needed to store all objects. The base class constructor now takes the size of the object and tracks the total size in a static variable. When the thread starts, the buffer is allocated and all objects are constructed inside this buffer. But more on this later. This technique has the advantage of allocating a single chunk of memory for a thread. It avoids memory waste of multiple allocations.

 
class ThreadLocalStorageBase
{
    static int m_buffer_size = 0;
    static TLSIndex m_tls_identifier;
    int m_index;
 
protected:
    ThreadLocalStorageBase( int object_size )
    {
        object_size = ( object_size + Alignment - 1 ) & ~( Alignment - 1 );
        m_index = m_buffer_size;
        m_buffer_size += object_size;
    }
 
    void * GetThreadLocalStorage() const 
    {
        char * local_storage_table = reinterpret_cast<char*>( TLSGet( m_tls_identifier ) );
        return &local_storage_table[ m_index ];
    }
};
 
template<typename T>
class ThreadLocalStorage<T> : public ThreadLocalStorageBase
{
    ThreadLocalStorage() : ThreadLocalStorageBase( sizeof( T ) )
    {
    }
}

OS TLS has the property to initialize the value to zero the first time a thread accesses it. As we support any class, we need to provide a way to construct the objects. We decided to initialize all variables before entering the thread function. If variables are not used frequently, a lazy initialization mechanism can be implemented. In Mojito, a class encapsulates the OS thread object. We have then the place to call this initialization. The best mechanism depends on the way you will use TLS variables.

In both cases, when the thread is about to end, the allocated memory must be freed. So a call to the TLS finalizer must be done. ( pthread allows a destructor function to be registered, so you can take advantage of it ). Once again, in Mojito, the thread class is taking care of it.

A question that might have arose : how do you have access to all variables spread across the code? The trick is quite simple : in the constructor, register the variable into a single-linked list. This technique ensures that as soon as main is entered, the linked list is complete. I discovered this trick in the command line option package of LLVM

As we have a class for each type, a base class has been created. It takes care of creating the list and provides the functions to initialize and finalize the variables. It also keeps track of accessing the thread local buffer.

class ThreadLocalStorageBase
{
    ...
    ThreadLocalStorageBase * m_next;
    static ThreadLocalStorageBase * m_first = 0;
 
public:
    ThreadLocalStorageBase( int object_size )
    {
        ...
        m_next = m_first;
        m_first = this;
    }
 
     static void StartupOnThreadEnter()
     {
        void * local_storage_table;
 
        local_storage_table = malloc( m_buffer_size );
        memset( local_storage_table, 0, m_buffer_size );
 
        TLSSet( m_tls_identifier, (void*)local_storage_table );
 
        ThreadLocalStorageBase * current = m_first;
 
        while( current )
        {
            current->InitializeObject();
            current = current->m_next;
        }
    }
 
    static void CleanupOnThreadExit()
    {
       ThreadLocalStorageBase * current = m_first;
 
        while( current )
        {
            current->FinalizeObject();
            current = current->m_next;
        }
 
        free( TLSGet( m_tls_identifier ) );
    }
 
protected:
 
    virtual void InitializeObject() = 0;
    virtual void FinalizeObject() = 0;
};

When the base class is in place, here are the modifications made to the subclass

template< typename T >
class ThreadLocalStorage : public ThreadLocalStorageBase
{
public:
    operator const T&() const
    {
        void * thread_local_storage = GetThreadLocalStorage();
        return *reinterpret_cast<T*>( thread_local_storage );
    }
 
    ThreadLocalStorage & operator=( const T & value )
    {
        void * thread_local_storage = GetThreadLocalStorage();
        *reinterpret_cast<T*>( thread_local_storage ) = value;
        return *this;
    }
 
protected:
 
    virtual void InitializeObject()
    {
        new( GetThreadLocalStorage() ) T();
    }
 
    virtual void FinalizeObject()
    {
        static_cast<T*>( GetThreadLocalStorage() )->~T();
    }
};

We now have a complete implementation of TLS that supports all types of object. But what if you wanted to initialize your variable to a specific value. This last modifications uses the container object to store a default value and use it when initializing the TLS instance.

The code is very simple. We just include a constant instance of the variable’s type, that we fill using the constructor. We then provide this instance to the constructor.

template< typename T >
class ThreadLocalStorage : public ThreadLocalStorageBase
{
    const T m_default_value;
 
public:
    ThreadLocalStorage( const T & default_value = T() ) :
        ThreadLocalStorageBase( sizeof( T ) ),
        m_default_value( default_value )
    {
    }
 
protected:
 
    virtual void InitializeObject()
    {
        new( GetThreadLocalStorage() ) T( value );
    }
 
    virtual void FinalizeObject()
    {
        static_cast<T*>( GetThreadLocalStorage() )->~T();
    }
}

This concludes this pair of posts on Thread Local Storage as we use them in Mojito. Comments are warmly welcomed. Special thanks to Bjoern Knafla for his comments.

The source code can be found here

Some compilers have an interesting property : __declspec(thread) for Microsoft cl.exe, __thread for Gcc. By using this declaration on a global variable, a copy of the variable is created for each thread. This is known as a Thread Local Storage (TLS). You can always use direct calls to the system’s API ( TLSGetValue and TLSSetValue for Windows, pthread_getspecific and pthread_setspecific for POSIX systems), but using automatic TLS variable is very comfy. As you may know, Mojito is available on several platforms, some of them does not provide those kind of variables. And if provided, those variables are limited to POD (Plain Ol’ data) variables. So we search for a solution that is close to what compiler can offer.

The solution presented here provides a mean to create such cosy variables, with a bonus : support for non-pod data.
To emulate the __thread specifications, the requirements are :
- The TLS variable should behave as a normal one.
- Memory should not be allocated if not necessary.
- Destruction of object must be properly done.
- Declaration should not be cryptic.

Solution : ThreadLocalStorage<MyClass> g_SomeThreadLocalStorageVariable;

The template class should have the following interface to conform to the specification :

template<typename T>
class ThreadLocalStorage
{
public:
    ThreadLocalStorage( const T & ); // Use to allow initialization of default value
    operator const T&() const;
    ThreadLocalStorage & operator=( const T & );
}

The major drawback of this technique : it only works in C++, as it uses operators of classes.
With this class, you can implement both operator in terms of getting and setting the value contains in the OS’ TLS. Generally, the TLS API provides 4 to 8 bytes of memory, generally to store pointers or basic types. This post will cover POD types, while complex types will be covered in the next post.

The code needed to fetch and store TLS is the same for each type of variable. It then makes sense to create a base class for this template.

class ThreadLocalStorageBase
{
protected:
 
    ThreadLocalStorageBase()
     {
         m_index = m_instance_count;
         ++ m_instance_count; // Not Thread safe
     }
 
     void * GetThreadLocalStorage() const 
     {
         void ** local_storage_table = reinterpret_cast<void**>( TLSGet( m_tls_identifier ) );
 
         return &local_storage_table[ m_index ];
     }
 
    int m_index;
    static int m_instance_count;
    static TLSIndex m_tls_identifier;
}

The sample code provided only use a single OS’ TLS to store all home-made TLS. The reason is quite simple : the number of TLS an OS provides vary greatly and can be as few as 2. In Mojito we decided that having multiple versions of this code is not usefull until we know the current implementation causes a performance or a memory problem. So we kept this code for all platforms. If you want to adapt it to your platform, you only have to override 4 methods TLSCreate, TLSGet, TLSSet, TLSDestroy.

You may have noticed the m_tls_identifier variable. This is where we store the OS TLS index. To initialize it, you must call ThreadLocalStorageBase::Initialize before your first thread starts. The ThreadLocalStorageBase::Finalize must be called after all threads have returned.

class ThreadLocalStorageBase
{
public:
    static void Initialize()
    {
         //Must be called before any thread is started ( not thread safe at all )
         m_tls_identifier = TLSCreate();
    }
 
    static void Finalize()
    {
        //Must be called after all threads have been stopped ( not thread safe at all )
        TLSDestroy( m_tls_identifier );
    }
}

The next question is how the OS’ Tls gets initialized. The first way, not shown here, is to poll the output of the OS’ Tls. If it equals to zero, it has not been initialized yet. It’s a lazy initialization. But it comes at the price of testing the value each time you access the variable. I have decided to pre-initialized it. Our threads are encapsulated in a class, so it’s easy to enforce the call to a function before giving hand to the user’s thread function. Same reasoning for deletion of memory. Not all systems allow the registration of destructors as pthread_key_create does. Again, using a class eases the use of a finalization function.

class ThreadLocalStorageBase
{
public:
    static void StartupOnThreadEnter()
    {
        void * local_storage_table;
 
        local_storage_table = malloc( m_instance_count * sizeof( void* ) );
        memset( local_storage_table, 0, sizeof( void* ) * m_instance_count );
 
        TLSSet( m_tls_identifier, (void*)local_storage_table );
    }
 
    static void CleanupOnThreadExit()
    {
        free( TLSGet( m_tls_identifier ) );
    }
}

The ThreadLocalStorage class is shown hereunder, even if I can already hear some guy shouting about aliasing rules

template< typename T >
class ThreadLocalStorage : public ThreadLocalStorageBase
{
public:
 
    static_assert( sizeof( T ) <= sizeof( void* ) );
 
    operator const T&() const
    {
        void * thread_local_storage = GetThreadLocalStorage();
        return *reinterpret_cast<T*>( thread_local_storage );
    }
 
    ThreadLocalStorage & operator=( const T & value )
    {
        void * thread_local_storage = GetThreadLocalStorage();
        *reinterpret_cast<T*>( thread_local_storage ) = value;
        return *this;
    }
};

The system is ready to go online. The sample code does not spawn thread, but the code displayed here shows how the system should be used :

ThreadLocalStorage<int> g_TestSmall;
// ThreadLocalStorage<std::string> g_TestBig; Will be possible in next post
 
void main()
{
    ThreadLocalStorageBase::Initialize();
    ThreadLocalStorageBase::StartupOnThreadEnter();
 
    g_TestSmall = 1234;
    int a = g_TestSmall;
 
    // Do some stuff, such as spawning threads    
 
    ThreadLocalStorageBase::CleanupOnThreadExit();
    ThreadLocalStorageBase::Finalize();
}
 
void thread_function() //signature depends on OS
{
    ThreadLocalStorageBase::StartupOnThreadEnter();
 
    g_TestSmall = 5678;
    int a = g_TestSmall;
 
    ThreadLocalStorageBase::CleanupOnThreadExit();
}

In the next post, I’ll explain how to handle complex class ( at least non-POD types ). This can be useful to store, for example, smart pointers. It will also talk about default values (other than zero of course).

While this technique is surely not to be used in hot spot of your code, it provides a simple way to have thread local variable. For example, storing the last error occurred, storing a non-thread safe logger, … Most of the use we have is in debug or for storing thread local memory allocator.

The sample code includes implementation for win32 and posix with an VC2005 and a xcode project. But our version of this code also runs on all consoles. Feel free to comment.

It’s been a while since I posted a technical article, but I’ve one nearly ready to go. It will talk about Thread Local Storage.

The slides from GDC can be download here

And it’s free.

Banana Scrum tool includes the following features:

• project backlog with tagging and intuitive sorting through drag and drop, which allows you to put there items that need to be done in the project

• sprint planning that allows you to assign items with tasks to the sprint

• sprint backlog where you can add tasks,expected time of doing it and sign them to team members

• impediment list where you can place encountered difficulties during the sprint and add it’s description

• file uploading for each item to be done in the sprint

• tags managing which allows you to add new tags and mark your tasks

• CSV export that will export your backlog items to CSV file

• burndown chart that shows you on a diagram if your sprint is feeling fine

• admin panel with users managing and project managing options

• different roles including Product Owner, Scrum Master and Team Member

• intuitive, easy to use user interface

• separate accounts with multiple projects and users

If you want to know more about Banana Scrum you can go here or watch a small presentation video here.

You wonder why this tool has such a silly name? Remember the Agile Manifesto? “Individuals and interactions over processes and tools”. This is just a tool, nice and useful but just a tool. They don’t want to get too pompous about it.

The idea is pretty simple : create a list of initialization function, sort it and call all functions in order. While simple, there is some problems to solve.

How to create this list? Simply by using a global helper variable. Another C++’s rule states that a global variable should not be stripped if its constructor has side effect. Interesting! So there is the code that we use

struct STATIC_DATA_FUNCTION_DECLARATOR
{
    STATIC_DATA_FUNCTION_DECLARATOR(
        STATIC_DATA_ENTRY * entry,
        const STATIC_DATA_FUNCTION function,
        const char * name,
        const char * predecessor_table,
        const char * successor_table,
        const STATIC_DATA_FUNCTION_TYPE type
        )
    {
        STATIC_DATA::AddFunction( entry, function, name, 
            predecessor_table, successor_table, type );
    }
};

Then just create a global variable of this type :

void initialize_function();
 
STATIC_DATA_FUNCTION_DECLARATOR function_declarator( 
    entry, &initialize_function, "ClassX", "ClassY", "ClassZ", 
    STATIC_DATA_FUNCTION_TYPE_Initializer );

While the previous class helps us creating the list, it is a mess to use. You have to write a method, then declare the global helper object. The result is an ugly code that nobody wants to remember. Usually, when I get there, I think it’s time for our friends macros to take care of that.

STATIC_DATA_InitializeType( TypeX, "TypeY", "TypeZ" )
{
    // Initialization code here
}
 
STATIC_DATA_FinalizeType( TypeX, "", "" )
{
    // Finalization code here
}

This code is preprocessed to :

void TypeX_InitializeType( void );
STATIC_DATA_FUNCTION_DECLARATOR TypeX_Initializer( 
    &STATIC_DATA_HELPER<TypeX>::Entry, &TypeX_InitializeType, 
    "TypeX", "TypeY", "TypeZ", STATIC_DATA_FUNCTION_TYPE_Initializer ); 
 
void TypeX_InitializeType( void )
{
    // Initialization code here
}
 
void TypeX_FinalizeType( void );
 
STATIC_DATA_FUNCTION_DECLARATOR TypeX_Finalizer( 
    &STATIC_DATA_HELPER<TypeX>::Entry, &TypeX_FinalizeType, 
    "TypeX", "","", STATIC_DATA_FUNCTION_TYPE_Finalizer ); 
 
void TypeX_FinalizeType( void )
{
    // Finalization code here
}

There is another little trick there. To ensure there is only one node per type (the node stores both the initialization and finalization functions), a template class that just defines one static variable is used:

template < class _TYPE_ >
struct STATIC_DATA_HELPER
{
    static STATIC_DATA_ENTRY 
        Entry;
};
 
template< class _TYPE_ >
STATIC_DATA_ENTRY STATIC_DATA_HELPER<_TYPE_>::Entry;

Inside STATIC_DATA::AddFunction, the entry is appended to the current list. So, if C++ rules are followed, when the program arrives in the main function, all the global helpers are initialized and the list must be complete. From there we call the function that will order the list and we call the initializer. Before exiting, we call the finalizers in reverse order.

int main( char* argument_table[], int argument_count )
{
    STATIC_DATA::CallInitializers();
 
    // Some usefull code
 
    STATIC_DATA::CallFinalizers();
}

The ordering criteria is simply the text that is passed in the macro. Several types might be passed. The second text contains successors of the initialization method.

STATIC_DATA_InitializeClass( SCRIPT, "MEMORY FILESYSTEM", "CONFIGURATION_SCRIPT" )
{
}

I’m not covering all aspects of the code, but you can find it with this working example. Enjoy!