Room 101

Gilad Bracha's blog. A place to be (re)educated in Newspeak

July 7th, 2024

I am Ozymandias


, and I am a computational notebook.  I also serve as my own design document, and my own documentation.  You can call me Ozy for short.

This is in itself an interesting aspect of Ampleforth - is this a practice to be emulated? Can we routinely get to a state where the code, its documentation, the design document and a working project are one and the same? This has the advantage of keeping the design document live and up to date as the code evolves. Maybe this is a way to get people to see the value of literate programming  - programming in the design document. Applied recursively at increasing levels of detail, it is a way to keep all documentation accurate, to integrate code with tests ...

I am also serving as a blog post about myself. 

This a bit of a stretch, as this post is primarily a living document representing Ozy's design and implementation, which itself is an experiment. There are bugs and quirks, but the fundamentals are my main concern, and they seem to hold up.

So, how does one use Ozymandias?

To create a cell, type in cell_X, where X is the name of a new cell, select the text and make it an amplet. Below are two cells, x and y, where x = 101 and y = x**2. If you edit x and re-evaluate, the value of y changes as well.




Unlike a traditional computational notebook, Ozy isn't restricted to a linear format. Cells can be placed side by side,  and they don't have to follow dependency order, so you can in fact construct a narrative that makes sense to the reader, different than the order  in which you constructed things.

Of course, this a legacy of Ampleforth's literate programming roots.

One can also nest computational notebooks within each other, or inside presentations (or vice versa).

How does Ozy work? 


The Newspeak IDE includes EvaluatorPresenters, widgets which allow us to evaluate expressions within the scope of a given object. Ampleforth allows us to embed these in documents.  If these evaluators are created on the document itself, they share a scope.  If we provide a distinct method to access the result of each such evaluator under a distinct name, and re-evaluate every time any evaluator is changed, the evaluators act as cells. This is the basic idea behind Ozymandias. All the rest is commentary, provided below.

A cell has an evaluator. The nested class Cell is a named wrapper around an EvaluatorSubject that is scoped to the document (the computational notebook). The document understands messages that are the names of cells.  Thus, any cell can reference any of the other cells. Of course, any circular definition will result in an infinite loop.

The document has a DNU that checks for messages of the form 'cell_X'; in that case, it adds a cell named X to the document. Adding a cell creates a fresh Cell instance, and also defines two new methods, X and cell_X.  The former returns the result of evaluating the cell. The latter returns the cell's presenter. 

A cell can provide a presenter by using its evaluator's presenter. DNU retries after adding the cell, which is guaranteed to succeed as the method was just added. It therefore returns a presenter for the new cell. Thereafter, the cell's value can be used by any non-dependent cell in its evaluator's code. simply by invoking X.

In more detail: DNU creates fresh cells by calling getCell:, which adds the cell to the cell table, cells (if it does not already exist). The two accessor methods, X and cell_X, access the cell via cells, and return its value and its presenter respectively.  Any accepted (or live) change to a cell's definition (i.e., its defining expression, as given in its evaluator) will generate a UI update. 

The UI update will cause the cell's presenters, instances of CellEvaluatorPresenter, to update.

CellEvaluatorPresenter is a custom subclass of EvaluatorPresenter that only shows one link, to its  last result. At this point, we can consider making cells into nested documents, so that they can be re-used. We might use an amplet like cellEvaluatorNamed: x initialExpression: e to achieve that, but that is getting fancy.

We face difficulties in that the state of the document is lost when saved. In particular, the contents of cells are lost. In fact, because the synthetic cell_X methods assume the cell is cached in cells, they do not function properly either. The latter point could be rectified by having them call getCell:, but the cell contents would still be missing. 

To address this, we use a nested class Cells (note the plural; this not class Cell). Upon save (using the addContentsUsingFolder: hook provided by the Document class) we create one method in Cells for every cell X in the document. The method returns the the cell's expression, as a string, and is named cell_X. In the case where there is no cell in cells, we ensure that the accessor methods for cells use at:ifAbsentPut: to place a cell based on the saved expression in Cells.

The space of conceivable situations is:

  1. There are no accessor methods and no value for the desired cell in cells. This the case when a new cell is created. In that case, doesNotUnderstand: creates the new methods and places a fresh cell in cells. The method is then invoked and we returns the new cell's presenter. After this, the following case holds for this cell name.
  2. There are accessor methods and there is a matching cell in cells. Here, the accessor methods run normally and extract their results form the cell stored in cells.  This occurs when a document refreshes (i.e, at every keystroke).
  3. There are accessor methods but there is no matching value in cells. This occurs when we load a saved document. In this case, the accessor methods  provoke the ifAbsentPut: clause, creating a cell based on the expression stored in Cells, and placing into cells. Afterwards, we again handle this cell name via case (2) above. To make things robust against a situation where the user manually removed cells when there were no stored expressions, Cells should have a DNU that returns a default value.
  4. There are no accessor methods but there is a corresponding value in cells. This situation should not occur, as a value is only introduced into cells in cases (1) and (2). In case (1), accessor methods are introduced prior to cell creation. In case (2), the methods already exist before the cell is created. But if we manually delete the methods, this situation can occur, and we may get debugging links (if the cell_X methods are removed) or error results (if the cell accessors themselves are removed).

An alternate scheme would have the cell contents listed in the document, from which one could derive a method and also create a showIt amplet. This would keep all data in the document itself,
but would require a special defineCell button that might be tricky wrt the selection. It would also require users to select and redefine the cells, and preclude live mode. Fancy objects require improvements to showIt as well.

Further thoughts: 
  1. How does this extend to extra features like sliders or color palettes that are best hooked up via ducts?
  2. What about displays of graphs etc. (should be simple, call a JS lib). 
  3. Should cells etc. be unique to Ozymandias, or should they be part of Ampleforth in general?
  4. Can we set up a spreadsheet in this way (in principle, for sure; in practice, might want a different presenter for cells). What about Alan Kay's example from SciAm?
  5. Cycle detection?
  6. The distinction between saving programs and state.
    The fact that this is a weekend's work is thanks to building on a system with the right capabilities: object scope and late binding, reflective invocation and change, reactive UI, document serialization. If we had support for snapshots, it might be even better.

A key weakness in this prototype is that it recomputes the entire notebook on every keystroke. We can fix this by having cells track their dependencies and dependents. During evaluation, a cell can record what cells it references, and then notify them that it depends on them (and of course, notify them when that stops being the case).  Cells can check, when evaluated, if their result has changed and if so notify their dependents to recompute. 

It's interesting to compare this to . By and large, we are on the same page. Their addition of laziness might be useful. We of course have advantages - a rich live IDE etc. On the other hand, the world just wants this stuff in Python (which it richly deserves).

Over time, I  expect to refine Ozy, making it, and the underlying layers, more robust and usable.