I'm happy to announce Aquameta, a web application development platform built entirely in PostgreSQL.

Why We Must Rethink Our Stack

1. Software Development Complexity Bottlenecks Human Progress

The complexity and inaccessibility of software development is inhibiting businesses, education and social progress.

In businesses, especially large ones, software has become the digital nervous system that allows them to operate at scale. But because only programmers can change the software, experimentation and evolution of business practices has actually become slower and more expensive. Most employees are powerless to make even simple software changes, and yet developers often don't have the domain expertise required to write good software for someone else's job.

Education is feeling it too. The demand for programmers is quickly outpacing supply, and for students, learning computer science is becoming more and more of an essential skill. Educators are well aware of this problem, but the complexity of programming makes it very challenging to teach.

Then there is social progress generally. End users have so many great ideas for software systems that could help us all collaborate with each other in deeper, richer ways. But to create them they either need the money or the skill to bring it into reality. Software complexity is a huge bottleneck for social innovation. Not every idea has a business model behind it, and there must be so many good ideas that could exist if programming was eaiser.

We believe that it is a social imperative, arguably one of the greatest demands of our time, to make software development more accessible to everyone.

2. Today's Web Is Better At Making Money Than Fun

Two-decades of profit-driven evolution have lead to the web we have today. But this outcome wasn't inevitable, and we can still change it. We have learned a lot since the inception of the web, and when we look at it knowing what we know now, we see some ways things could be a lot cooler. Here are a few.

Conflation of Data and Presentation

Today's web is made of "pages", which conflate the data layer with the presentation layer. When we want an article or video, rather than being able to download just that content, we have to download the entire "page" that the content is on, often times containing ads, cross-promotions, etc. A more advanced architecture would give the user the power to control what we downloaded selectively.

Trending Towards Centralization

The Internet was designed as a peer-to-peer network, where every computer could communicate directly with every other computer. Yet the way we use the web today is primarily through centralized silos of information. All our computers talk to a really big computer, say Facebook, who mediates the interaction. If I want to share a photo with my mom, there is no architectural reason why any central party needs to be the mediator.

But it's about more than just privacy or exploitation of the commons. Centralized systems are BORING. The early days of the web were honestly more exciting, more raw, more wild-wild-west. We need to get back to that vibe.

Complexity of Systems Invention

Once upon a time there were no end-user tools for contributing to the web. "Users" just wrote HTML by hand. We've come a long way since then with GUI editors, blogging platforms, social networks etc, but still today there is a wall between the technical haves and have-nots, and that's that it is very difficult for a "mere mortal" to build a data-driven web application. In the web we invision, building data-driven web apps is something accessible by everyone.

We need to rethink programming and the web, to fix some of the architectural short-comings and open up a new space full of new possibilities, and new problems to solve.

Approach

At the foundation of Aquameta's reimagining of programming and the web are two central ideas, datafication and visualization.

1. Datafication

The complexity of our stack is pretty daunting when you list it all out. Some of this complexity is unavoidable, computer science is just fairly complex; but there is another swath of knowledge and skills that has little to do with computer science and a lot more to do with just the diversity of our programing tools. Consider /etc, for example.

This diversity and complexity isn't a "problem". In fact, one could argue that it is an essential ingredient of the primordial soup from which so much innovation has emerged. However, when you take a step back and look at it, it seems pretty unnecessary, and it does make it harder for beginners to learn.

Our first step towards making programming easier is to use the database to model the development stack itself.

In Aquameta, the entire stack is accessible as data, so diverse tools can share a common information model. It puts the database at the foundation of the developer experience. We use PostgreSQL as a kind of "infokernel".

It turns out PostgreSQL is really good at this. Between foreign data wrappers and the various procedural languages, there isn't much that PostgreSQL can't make look like data.

A datafied stack makes a common interface to all these different tools, which eliminates a ton of complexity.

2. Visualization

Visual interfaces are a lot easier for beginners to use, and writing visual interfaces against a datafied stack is ridiculously easy and fun. As web programmers, we have a ton of experience writing interfaces to databases, so when the whole dev stack is data, we can apply that skill towards experimenting with user interfaces that users can quickly understand.

If Aquameta is a success, we'll see an explosion of different visual programming tools, and a vast diversity of competing approaches to making programming easier.

The Architecture

Aquameta has eight core layers, each of which is a PostgreSQL schema:

• meta - a writable system catalog for PostgreSQL
• bundle - a snapshotted version control system for data
• filesystem - models the filesystem as relational data, and in reverse, makes PostgreSQL visible as a filesystem
• event - a pub/sub data change event system built atop PostgreSQL's NOTIFY system
• www - a HTTP request handler that authenticates against the PostgreSQL user system and serves up arbitrary text/binary resource in the database, files from the file system, a REST interface to the database, and websocket events
• p2p - Mechanisms for doing transmitting data between one end-user annd another across NATs
• widget - modular web user interface components
• semantics - a space to decorate the schema with "meaning", human identifiers, crud widgets, etc.

Project Status

Aquameta is the result of several years of work by Eric Hanson and the folks at Aquameta Labs. It's currently pre-alpha. We haven't even done what we would call a 0.1 release. We're releasing it primarily for architecture nerds to get some feedback and contributions.

Our plan is to publish a writeup of each module in the coming weeks, starting with meta and moving up. We'd love to hear what you think, and patches are welcome.

http://github.com/aquametalabs/aquameta