Probably Dance

I can program and like games

Category: Programming

Quickly Loading Things From Disk

Loading things from disk is a surprisingly unsolved problem in C++. If you have a class that you want to save to JSON or binary and load back later, you’ll probably end up writing custum parsing code. Solutions like Protobuf or Flatbuffers generate custom classes that are… how should I put it… bad code. They force you to work in a parallel type system that’s not quite as powerful as C++ and that doesn’t allow you to write custom member functions. You pretty much have to wrap anything that they generate. Boost serialization is a better solution in that it allows you to load and save existing classes, which allows you to keep your code quality higher, but it’s unable to write human readable files. The default implementation that it provides is also surprisingly slow. I blame it on the fact that it loads using std::istream.

Based on frustrations with existing solutions I decided to write something that can

  1. read and write existing, non-POD classes. No custom build step should be required
  2. write human readable formats like JSON
  3. read data very quickly (I care less about how fast it writes stuff. I work in an environment where there is a separate compile step for all data)

I came up with something that is faster than any of the libraries mentioned above.

Read the rest of this entry »

A surprisingly useful little class: TwoWayPointer

In graph like data structures (explicit or implicit) there is a problem of memory safety when you remove a node. You have to clean up all the connections pointing to you. If your graph structure is explicit this is easy, otherwise it can often be annoying to find all pointers pointing to you. A while ago I thought that you could solve that by introducing a two way pointer, which has to be the same size as a normal pointer, but which knows how to disconnect itself on the other side, so that you don’t have to write any clean up code.

The reason why it turned out surprisingly useful is that if both ends of the connection know about each other, I can make them keep the connection alive even if one of them is moved around. So if one of them lives in a std::vector and the vector reallocates, the TwoWayPointer can handle that case and just change the pointer on the other end to point to the new location. That can dramatically simplify code.

One example is that I’m now using it in my signal/slot class which I’ve also included in this blog post. It allowed me to simplify the signal/slot implementation a lot. Compare it to your favourite signal implementation if you want. There is usually a lot of code overhead associated with memory safety. That all went away in this implementation.

Read the rest of this entry »

Ideas for a Programming Language Part 5: More Flexible Types

Type systems tend to be too rigid. And dynamic typing just throws the baby out with the bathwater. So here are a few things I want:

1. I want to add information to an object at compile time without having to give it a new type. Is an integer in seconds or kilometers or kilometers per hour? Is a string SQL escaped or URL escaped or just plain text? Is a pointer known to not be zero? I also want to write logic based on this. If I multiply seconds with kilometers per hour, I want a result that makes sense. Most importantly, I want to do this easily: There is nothing stopping me from having a “url_escaped_string” type in C++, but then that string doesn’t work with any of my existing functions. I can add an implicit conversion operator, but then I lose my compile time information as soon as I use a function that accepts only one type of string. Or everything has to be templates and everything has to be duck-typing.

So what I envision is that every object has runtime properties and compile time properties. So a string might have a runtime property called size(), and it might have a compile time property called language() where I can set the language to be “SQL” or “URL” or similar and could then make sure at compile time that everything is escaped correctly. An int would not have any runtime properties, but it would still have compile time properties like its unit of measurement. We’ll start with that and then we’ll see what other features we’d want from that. (like functions being able to write conditions for their input parameters where they check the compile time properties of those objects and similar)

Read the rest of this entry »

Ideas for a Programming Language Part 4: Reactive Programming

Reactive programming has been around for a long time. It’s a paradigm oriented around data flows. The usual example is a spreadsheet: If cell B refers to cell A, then cell B updates automatically when I change the value in cell A. If ten cells refer to cell A, then all those ten cells change automatically. This is as opposed to C++, where if I compute ten variables using variable X, then if I change variable X, I have to manually write code to make sure that the other ten variables get updated.

What’s changed recently is that there are a few interesting approaches that make this paradigm more widely useful. Functional Reactive Programming (FRP) is a useful approach which is spreading, with popular libraries available for pretty much all languages. For an illustration of FRP and how it can be used in an imperative style I recommend reading Deprecating the Observer Pattern.

What really convinced me though that a lot more code should be reactive is a javascript library called Knockout. Which makes it so easy to write reactive code, you’d be stupid to ever write manual state propagation code.

Read the rest of this entry »

Recommended Link: Inventing on Principle

https://vimeo.com/36579366

This talk was influental recently but I feel like far more people should follow this advice. The philosophy part is good, but it’s really about the demos.

Read the rest of this entry »

Ideas for a Programming Language Part 3: No Shadow Worlds

This post is inspired by this text by Gilad Bracha. I recommend that you read it before reading this blog post. The idea of Shadow Worlds is that people keep inventing these constructs that are powerful enough that you want to use them as programming languages, but they are not powerful enough to have the features of high level programming languages that make programming enjoyable.

Before reading that blog post I knew that I wanted a macro system for my language. Looking at my old notes I had written down that a macro system is necessary, and that templates are not enough. However I wanted a type safe macro system from modern languages like Nim or Rust. Gilad Bracha’s blog post made me realize that you actually never want macros because they are just another Shadow World. Instead you want to do everything in your high level language.

Read the rest of this entry »

Recommended Link: Deprecating the Observer Pattern

A note about this post: I sometimes feel like I should have a Twitter account just to share random links, but I’d also like something more organized than Twitter. So I think I’ll just post short blog posts here every now and then which are basically just a link and a comment about the link. Let’s start with Deprecating the Observer Pattern:

http://infoscience.epfl.ch/record/176887/files/DeprecatingObservers2012.pdf

One of my favourite papers. After you’ve read it here’s my comments:

Read the rest of this entry »

Ideas for a Programming Language Part 2: A more liberal call stack

One thing that came out of playing around with coroutines is that I eventually realized that callstacks are a bit annoying. It’s one of these things where it’s difficult to convince people because callstacks are equivalent in power to any other form of organizing your program flow. But I think that callstacks constantly force us into solving two puzzles: 1. How to split some sequential, blocking, code up into several smaller pieces of non-blocking code because the function has to return quickly. 2. Where to store an object (or keep track of an object) because it is long-lived but the current scope has to be short-lived.

If you’ve been programming for a while you are really good at solving both of these puzzles because you solve them all the time. Which is another reason for why it’s difficult to convince people that it’s worth looking into alternatives: If you are a programmer today, these don’t seem that complicated to you.

But I want to investigate how much can be gained by looking at different approaches to call stacks. I think the potential gains are huge, because this is such a core piece of programming.

Read the rest of this entry »

Ideas for a Programming Language Part 1: Source Code in Database

My main motivation for wanting to write a programming language is that I believe I can make programmers much more productive. Somewhere in the archives of unfinished blog posts for this website here I have a very long post about what language features and coding practices I think contribute to programmer productivity, (and what harms productivity) but I believe that in the long term the one most important contributor is the quality the tools that we use to program.

And “tools” is a very vague word. It includes everything from source control to text editors to test frameworks to debuggers to refactoring helpers to code completion to compiler warning messages to static analysis to profilers, diff, make and many more categories. All of this surrounding infrastructure that we use around coding actually has a much bigger impact on productivity than which language we pick. Which makes sense because those tools are usually written by someone not directly involved in the language. And if they took the time to write the tool, you bet it will have a big impact on their productivity. So I want to make a language that will have the best tools ever. I think that’s actually not too difficult. It seems that nobody has had that ambition before.

And for that my language will store its source code in a database instead of text files.

Read the rest of this entry »

plalloc: A simple stateful allocator for node based containers

Allocators in C++ are awkward classes. Allocators are usually zero size to make the containers small, and they have no state because that used to be difficult to do in C++. But nowadays we don’t really care about the size of our containers (we care about the size of the content of the containers) and since C++11 we have support for stateful allocators.

The problem I’m trying to solve is that node based containers have bad cache behavior because they allocate all over the place. So I wrote a small allocator which gives out memory from a contiguous block. It speeds up std::map and std::unordered_map. It’s called plalloc because I think this is a pool allocator.

Code is below:

Read the rest of this entry »