Notebook state

Programmatic access to notebook cells and outputs. 
bridge_cfg.auto_show = True
bridge_cfg.auto_id = False
bridge = get_bridge(show_logger=True, wait=5)

NBStateFeedback

Simple visual feedback of notebook state. We manage the notebook state In the front-end and use this helper to show a little feedback of state changes.

If yuou’re reading this in VSCode-ish environment, ensure the extension developed in nbinspect-vscode is enabled.

If you want to know how the notebook’s state gets here in real-time, take a look at the packages folder.

# NBStateFeedback.show(feedback=False, hide=True,debug=False)
NBStateFeedback.show()

Note that the state data is still in the front-end (JS-land). This function is just a visual feedback of state changes, but we haven’t transferred yet the state to the kernel (python-land). We need to have something (the bridge) in the front-end that can transfer the state to the kernel. For that, see next sections.

VSCode weirdness (again): Bridget uses an extension to capture notebook state. In Jupyter environments, the extension is triggered when a notebook is opened, monitors state changes, and sends notifications to the frontend, waiting for a widget to retrieve them. Check the JavaScript console to see the extension’s traces.

In VSCode, the extension is also activated when a notebook is opened. However, the front end of the extension—Renderer in VSCode Notebook Extension API parlance—is isolated within a webview, an iframe-like object that the extension can only communicate with via the a very limited messaging API. Renderers are the only means by which extensions can communicate with the JavaScript running in the webview. And renderers can only be triggered by a cell displaying specific MIME types at least once. Therefore, since VSCode only renders its output when a visible cell is executed, the extension’s renderer can only obtain the notebook state relayed by the main extension when the cell is run interactively or is already in the notebook and visible. Fortunately, this only needs to happen once; you can delete the cell or its output, and the extension will continue to work. VSCode’s foibles.

Keep this in mind when running commands like Execute Above Cells or Execute All Cells or similar. If the above cell is already displaying its output, that’s fine; if not, the extension renderer won’t be triggered until it appears, the state won’t be passed to the front end, and the rest of the notebook won’t work properly.

print(19)
19

After executing above cell, you’ve noticed a pink flash. We’re monitoring state changes in real-time.

Clear the output of the above cell. Open the details and look for cell #19 (if you haven’t added any new cells above it). Check that it doesn’t have any outputs.

Run the cell again and check to see that it now has outputs.

Create a new cell below this, source or markdown, and write something. Notice that we’re updating the state of cell inputs not just outputs.

NBStateFeedback.update(feedback=False)

With feedback disabled, the details disclosure in not shown.

NBStateFeedback.update(feedback=True)
NBStateFeedback.update(debug=False)

debug affects javascript dev console logging.

print(29)
29
NBStateFeedback.update(debug=True)
NBStateFeedback.show()
Unable to display output for mime type(s): application/x-notebook-state+json
NBStateFeedback.hide()

Though we’ve hided the feedback output, Bridget is still monitoring and capturing state changes, as you can see in the dev console. In fact, once activated by showing the feedback output, it’s not possible to deactivate it.

NBStateFetcher

A bridge plugin to retrieve notebook state from the front-end to bridge.state.

Simple plugin to grab the notebook state from the front-end and return it to Python where we really want and need it. And where it should have always been, imho. How much unnecessary pain has the MVC pattern done!

bundled(nbstate_js)();

source

NBStateFetcher

 NBStateFetcher ()

Bridge plugin that retrieves notebook state from the front-end

bridge.logger.show(clear=True)
bridge.add_plugins(fetcher := NBStateFetcher(), wait=3)

l = 80
test_is('fetcher' in bridge.plugins, True)
test_is(len(bridge.state.cells) in (0, l), True)

WARNING: the extension debounces state notifications to avoid excesive throughput. When running Execute Above Cells or Execute All Cells or similar commands, the extension will group state notifications and the cell below will probably fail. If that’s the case, simply run the cell again.

# cell 39
test_eq(len(bridge.state.cells), l)
uri = urllib.parse.urlparse(bridge.state.nbData['notebookUri'])
if in_vscode_notebook(): test_eq(uri.path, __vsc_ipynb_file__)  # type: ignore
bridge.state[39]
NBCell@39
  • idx: 39
  • cell_type: markdown
  • source: WARNING: the extension debounces state notifications to avoid excesive throughput. When running `Execute Above Cells` or `Execute All Cells`…
  • id: X54sZmlsZQ==
  • metadata: {'brd': {'id': '5d660c84-66e2-41ab-b3c5-c2e67f16fc7d'}}

Note after running previous cell, the state has not yet been updated. Check that it has no outputs or the outputs are old. We’ll see how to access the state of the current cell below.

# cell 41
bridge.state.cells[39]
NBCell@39
  • idx: 39
  • cell_type: markdown
  • source: WARNING: the extension debounces state notifications to avoid excesive throughput. When running `Execute Above Cells` or `Execute All Cells`…
  • id: X54sZmlsZQ==
  • metadata: {'brd': {'id': '5d660c84-66e2-41ab-b3c5-c2e67f16fc7d'}} 
bridge.logger.show()
print('test5')
test5
display(HTML('<div>test6</div>'))
test6

NBState

Simple wrapper around NB + NBStateFetcher & NBHooks bridge plugins

source

NBState

 NBState (source:Union[__main__.StateProvider,Mapping,NoneType]=None,
          *bridge_args, plugins=None, **bridge_kw)

Inherit from this to have all attr accesses in self._xtra passed down to self.default

source

StateProvider

 StateProvider (*args, **kwargs)

*Base class for protocol classes.

Protocol classes are defined as::

class Proto(Protocol):
    def meth(self) -> int:
        ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).

For example::

class C:
    def meth(self) -> int:
        return 0

def func(x: Proto) -> int:
    return x.meth()

func(C())  # Passes static type check

See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as::

class GenProto[T](Protocol):
    def meth(self) -> T:
        ...*
state = json.loads(Path('../packages/nbinspect-vscode/test/outputs.json').read_text('utf-8'))

nb = NBState(state)
display(nb.mds)
nb.codes
(#2) [0,23]
(#40) [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20...]
nb = NBState(show_logger=True, wait=5)
test_eq(nb.source, get_bridge())
if len(nb.cells): display(nb[50])
NBCell@50
  • idx: 50
  • cell_type: code
  • source: nb = NBState(show_logger=True, wait=5) test_eq(nb.source, get_bridge())
  • id: Y101sZmlsZQ==
  • metadata: {'brd': {'id': '14138586-2875-4e27-9c03-200e102a591d'}}
    • outputs
      0
      • output_type: display_data
        • data
        • text/html: <div id='brd-logger_4-1764771587' class='brd-logger' style='width: 100%; max-height: 200px;'></div>
        • text/plain: <bridget.helpers.HTML object>
      • metadata: {'transient': {}}
  • execution_count: 40 
list(shortens(nb[40:50], 'r', 80))
['{\'idx\': 40, \'cell_type\': \'code\', \'source\': "# cell 39\\ntest_eq(len(bridge.state.…',
 '{\'idx\': 41, \'cell_type\': \'markdown\', \'source\': "Note after running previous cell…',
 "{'idx': 42, 'cell_type': 'code', 'source': '# cell 41\\nbridge.state.cells[39]', …",
 "{'idx': 43, 'cell_type': 'code', 'source': 'bridge.logger.show()', 'id': 'X61sZm…",
 '{\'idx\': 44, \'cell_type\': \'code\', \'source\': "print(\'test5\')", \'id\': \'X62sZmlsZQ==…',
 '{\'idx\': 45, \'cell_type\': \'code\', \'source\': "display(HTML(\'<div>test6</div>\'))", …',
 "{'idx': 46, 'cell_type': 'markdown', 'source': '# NBState\\n> Simple wrapper arou…",
 '{\'idx\': 47, \'cell_type\': \'code\', \'source\': "#| export\\n\\n@runtime_checkable\\ncla…',
 '{\'idx\': 48, \'cell_type\': \'code\', \'source\': "state = json.loads(Path(\'../packages…',
 "{'idx': 49, 'cell_type': 'code', 'source': 'display(nb.mds)\\nnb.codes', 'id': 'Y…"]
list(shortens(nb.find('class').attrgot('source'), 'r', 80))
["#| export\n\nNBSTATE_MIME = 'application/x-notebook-state+json'\n\nclass _NBStateFee…",
 '#| export\n\nclass NBStateFetcher(BridgePlugin):\n    "Bridge plugin that retrieves…',
 '#| export\n\n@runtime_checkable\nclass StateProvider(Protocol):\n    @property\n    d…',
 "list(shortens(nb.find('class').attrgot('source'), 'r', 80))"]
# find_me
# this cell2
# nb.update()
nb.find('# this cell2').attrgot('source')
(#1) ["# this cell2\n# nb.update()\nnb.find('# this cell2').attrgot('source')"]

Cell id

For this to work, bridge must be loaded with the NBHooksPlugin (NBState takes care of all that) .

# really hate stupid wiggly reds
__cellinfo__:AD
print(f"{__cellinfo__.cell_id!r}\n{__cellinfo__.source!r}")
test_eq(__cellinfo__.source[:29], 'print(f"{__cellinfo__.cell_id')
'Y114sZmlsZQ=='
'print(f"{__cellinfo__.cell_id!r}\\n{__cellinfo__.source!r}")\ntest_eq(__cellinfo__.source[:29], \'print(f"{__cellinfo__.cell_id\')'
# cell x
idx = nb.find('# cell x')[0].idx  # type: ignore
test_eq(__cellinfo__.cell_id, nb[idx].id)
nb[idx]
NBCell@61
  • idx: 61
  • cell_type: code
  • source: # cell x idx = nb.find('# cell x')[0].idx # type: ignore test_eq(__cellinfo__.cell_id, nb[idx].id) nb[idx]
  • id: Y115sZmlsZQ==
  • metadata: {'brd': {'id': 'dbdca1cc-dcde-4bc3-9738-c1e14cd72069'}}
    • outputs
    • execution_count: None

    this

    this marks the finale of the first part of Bridget.

    We now have all the pieces to build the bridge. - fasthtml & other helper scripts - loader of arbitrary JS code - notebook state fetcher - nbstate and nb hooks

    Next steps will be to add tools to edit notebook outputs.

    source

    this

     this (idx:int|None=None)

    Current cell if idx is None, or cell at idx from current cell upwards. Raises if not found.

    bridge.logger.show(clear=True)
    c = this(); c
    NBCell@66
    • idx: 66
    • cell_type: code
    • source: c = this(); c
    • id: Y123sZmlsZQ==
    • metadata: {'brd': {'id': '73daee75-5371-4c72-9211-52443b4b2bf7'}}
      • outputs
      • execution_count: None 
      c
      NBCell@66
      • idx: 66
      • cell_type: code
      • source: c = this(); c
      • id: Y123sZmlsZQ==
      • metadata: {'brd': {'id': '73daee75-5371-4c72-9211-52443b4b2bf7'}}
        • outputs
          0
          • output_type: execute_result
          • execution_count: 54
            • data
            • text/plain: {'idx': 66, 'cell_type': 'code', 'source': 'c = this(); c', 'id': 'Y123sZmlsZQ==', 'metadata': {'brd': {'id': '73daee75-5371-4c72-9211-5…
            • text/html: <style>details ul { list-style-type:none; list-style-position: outside; padding-inline-start: 22px; margin: 0; } details .string { color: #2…
            • text/markdown: > code ```json { 'idx': 66, 'cell_type': 'code', 'source': 'c = this(); c', 'id': 'Y123sZmlsZQ==', 'metadata': {'brd': {'id': '73…
          • metadata: {}
      • execution_count: 54

      this is meant to be used interactively (as most of Bridget is).

      The main function of this is to get the current cell, or more so, the current cell’s index, and get your bearings in the current notebook structure.

      Note that due to the client-server nature of the Jupyter notebooks, code running in the kernel can’t possibly access the cell currently being executed. The cell state is maintained byt the front-end, not the kernel. The kernel in fact knows nothing of notebooks or their structure. We can aspire at most to get the cell’s index and source. The output is not yet determined, though Bridget will capture it and you can access it after the cell has run, not during the cell execution. You can find the cell with NBState afterwards.

      Also be aware, as always, that batch commands like “Execute Above Cells” and similar are implemented very differently by Jupyter Notebooks vendors. The cells in the run queue may start running in order (or not, I’ve seen ships on fire off the shoulder of Orion), but not necessarily finish up in that order. Bridget may receive the cells in any order and therefore, the notebook state is not settled until the end. In general, in Jupyter Notebooks, cells that depends on outputs of other cells will surely fail when not running interactively. This is known issue (and a desing flaw imo, that can be resolved simply by the front-end sending the cells in order to the kernel and waiting for each one to fisnish before sending the next one) and one of the reason Marimo and other Jupyter modern alternatives exists.

      this(-1)
      NBCell@68
      • idx: 68
      • cell_type: markdown
      • source: `this` is meant to be used interactively (as most of Bridget is). The main function of `this` is to get the current cell, or more so, the c…
      • id: Y125sZmlsZQ==
      • metadata: {'brd': {'id': '93341472-e8c3-44ed-8a54-24e71d861fbc'}} 
      this(-3).source
      'c'
      this(FIRST).source
      '#| default_exp nb_state'
      this(LAST).source
      "#| hide\n#| eval: false\n\nif FC.IN_NOTEBOOK:\n    BUNDLE_PATH = bundle_path(__name__)\n    for f in ['nbstate']: bundled(BUNDLE_PATH / f'js/{f}.js')()\n    nb_path = '21_nb_state.ipynb'\n    nbdev_clean(nb_path)\n    nbdev.nbdev_export(nb_path)"
      this(1).source
      '# get_nb'

      get_nb

      source

      get_nb

       get_nb (*args, show_feedback:bool=False, logger:NBLogger|None=None,
         show_logger:bool=False, lnks:dict[str,FT]|None=None,
         esms:dict[str,str|Path]|None=None,
         plugins:Sequence[BridgePlugin]|None=None,
         kwplugins:dict[str,str]|None=None, wait:int=0,
         summary:bool=False, factory:Callable[...,Any]|None=None,
         timeout:float=10, sleep:float=0.2, n:int=10,
         show:Callable[[bool],None]|None=None)
      Type Default Details
      args VAR_POSITIONAL
      show_feedback bool False
      logger NBLogger | None None
      show_logger bool False
      lnks dict[str, FT] | None None
      esms dict[str, str | Path] | None None
      plugins Sequence[BridgePlugin] | None None
      kwplugins dict[str, str] | None None
      wait int 0 seconds to wait for plugins/links/modules to load
      summary bool False
      factory Callable[…, Any] | None None
      timeout float 10
      sleep float 0.2
      n int 10
      show Callable[[bool], None] | None None