jkomoros / code-sprouts Goto Github PK

With a GUI... but later also running fully deployable

• Document in README
• Next step: pretty print conversation
• then: provide the user's response to the conversation
• Add serve:generate under serve once it stops re-running a billion times
• Make the message box show up at the bottom no matter what, and the messages scroll up from the bottom.
• Make sure it is deployable (the build / tsc--watch might collide in build dir)
• Add back in devCompose Redux store
• Image attachment drop target
• Since patch/state updates before messageForUser, refactor it to show the state in zippy before the whole message is done (this will be a massive refactor and change the whole signalling architecture)
• Refactor the whole app into sprout-viewer so it can be embedded in another view easily (look at prompt-garden)
• An error dialog
• Mobile formatting
• Support lastModified information in directory.json
• Bug: switch from one sprout type to another and then ask a question. It streams back double
• Bug: switch from one sprout type to another while answer streaming back in. Then switch back. Something gets wedged (unclear if you have to do it in in the middle of the streaming or if it still happens after)
• Keyboard shortcut to send message
• Add an animated demo image to README
• SEO indexing like card-web does
• A view-source button in the header that pops a dialog (starts out just printing out the JSON in a pre)
• Create a proprietary downstream fork that also allows users to create their own hosted sprouts for others to use
• Why does codenames keep getting recompiled?
• A prompt dialog
• Remember other sprout paths that have been added
• First run should explain how it works and accept the openai key
• watcher.ts should only copy or generate the specific thing that changed (or batch up a few changes over a ahlf-second and then run mulptile paths at once, running each command only a single time)
• Show a warning if the current sprout isn't valid
• Allow adding a folder of sprouts
• Have a --watch for generating directory listings and compiling sprouts
• Fix bug where lastState is somehow duplicated, if you add an image, and then once it's down processing, if you expand state you'll see the same state object twice.
• Support remote sprouts in fetcher-node.ts too
• Explain what's happening when it says it's still updating even when no text is streaming in
• Don't show image button if not alllowsImages.
• Handle very long / scrolling conversations
• Loading icon should spin when streaming.
• Animated loading indicator when a result is streaming
• When uploading images, the attach button should say the filename that is attached.
• disabled style for buttons that are loading
• Resize images clientside to be smaller when loading (use pica)
• Add back in lazy reducers (or remove the package)
• Don't allow drag/dropping images if the attach button are disabled
• Show a message about dropping images when hovering with images
• Allow loading remote sprouts (will need a way to represent them in the URL parameter as well as fetching)
• Allow loading sprouts from the URL
• Allow selecting sprouts from a dropdown
• Allow the user to see the full debug output behind a zippy
• A regenerate last message button
• Make sure that the gpt-tok is only loaded on demand.
• Persist conversations
• Render title/description of sprout in UI
• Allow a theme color / icon for sprout
• Resize image to be smaller in browser
• Host at a domain
• A sprout:compile that compiles all local sprouts (and is run before deploy)
• Have a warning that a remote sprout doesn't see your key
• Have a warning that the remote sprout isn't vouched for
• Check that the sprout folder names don't have dots in them.
• Allow markdown output (if the runner says it can) (and tell the prompt it may)
• Allow the runner itself to say whether it can support images
• Some kind of feedback loop for users to give feedback on the response and automatically tweak the instructions
• (separate issue) Allow the typescript schema to be provided in plain english and be compiled to typescript
• Support images
• Have fetcher cache results
• tab title based on sprout
• A way to add all sprouts at a given remote directory

Currently it has to fetch multiple files. Maybe it first tries to fetch compiled.json and if not builds it itself.

Building might require some LLM calls, so in some clients it will only work with compiled ones, to avoid expensive rebuilding.

E.g. "reaching out to a REST endpoint"

Show a dialog to a user for new action types for new sprouts.

In the future, maybe have an optional suspicious signals clearing house?

• Get image upload working
• Use images for codenames example
• A "create a story for a young kid" example that keeps track of things like the kid's name, the story type, etc.
• Create a "Let's Find Out" bot that answer's kids factual questions like "Do geese eat grass?" using simple language and questions at the end to encourage the kid to reflect and encourage their curiosity

• Fix as any in sprout.conversationTurn around patch
• Persist state across sessions

Now that we have #21 , it actually makes sense to add back in writeSprout and deleteSprout, so the fetcher htat does the fetching also knows how to do the writing.

DataManger will then become a much smaller class and might wither away entirely.

A kind of conceptual undo of #20

Related to #19

That's instead handled in the dataManager layer entirely, which wirtes into localFilesystem

We don't save compiled files out and the tooling fetches the compiled data and saves out instead of assuimign the sprout will.

Related to #19

Related to #3

• fetcher.mayWriteFile(path) : Promise
• Rename sprout for viewer to view, since the common case will have lots of view/sprouts/username/sproutname/ in the URL, and two sprouts in a row is duplicative
• Get partial compilation working, where when a file changes, only the parts that have changed are updated.
• Make a app/data_manager.ts that has the API key managing methods to start
• fetcher.readFile and writeFile should have a localDir property, and if the path has that prefix, use the filesystem API as described in the ORG-DESIGN.md
• The webapp should be told that sprouts in localDir exist when booted.
• Add DataManager.writeSprout(packaged) that calls into fetcher.writeFile. Fetcher.writeFile should not overwrite the fileinfo if the content is byte-for-byte idententical.
• have a selectCurrentSprout also take a selectLocalDataForCurrentSproutName, which in most cases will return null, but in some cases will share the data. This helps it figure out when the currentSprout needs to be refetched.
• Add a cancel button to the Sprout dialog. Only show it if the dialog is editable.
• ts syntax highlighting in the schema.ts box
• The apptitle on first run before a sprout is loaded is just a dash
• When the editor is open, add something in the hash so you can deep link others into the configuration for the app
• Make tabs in the schema box insert a tab, not select the next control
• Keep track of edits have been made to a sprout and warn if cancelling without saving.
• have the editable sprout be read out in an expandable drawer above the conversation (so when it's open it just takes up more space. And recompile and rerun them as you edit so you can interact with it)
• Allow deleting a sprout (that was created or added)
• When a sprout is opened for editing, snapshot the compiled output. Then make modifications to that compiled snapshot. Then when the change is submitted, create a packagedSproutFromCompiled and then call data.writeSprout(). That layer will call writeFile, which will write into the in-memory filePath structure and also each time it writes, commit to localStorage. And then modify writeFile so if the content is precisely the same, lastUpdated isn't changed.
• Figure out a way to make edits to the sprout.
• Wire through events to update sprout
• Add a SproutDefinitionDialog

• Allow a user to cancel in the middle of waiting for a token from the bot.
• Fix the bug where switching from one sprout type to another leads to duplicate messages
• Render multi-line responses wrapped in <p>
• The limerick bug is when you ask the limerick example for a limerick, the result has duplicates in it. The reason is that the result includes newlines, and they stream in as \ and then n. That means the previous and next don't diff cleanly, leading to it returning the whole string, which leads to duplicating it.
• Stream in characters for the user message like claude does it: one character at a time. Chunk the new characters in a bufffer, and them stream them out at a rate that aims for consistency based on back pressure and rate of new chunks coming in. This gives a very slightly delayed but much smoother experience

With more than just text/images interaction.

You can imagine the bot being able to respond with prebaked responses for the user to respond with.

Allow multiple bots to work together.

Taking this even further, you can imagine a more formal notion of threads as described here: https://docs.google.com/document/d/1nmn3xi_LyR_c2inNsDNGkfUGA1lxRKyV7zHfNTNdddo/edit#heading=h.n3w7p91fkufp

• sprout.json version --> format
• Add sprout.json semVar version for the sprout to define its own version
• forkedFrom : SproutName
• Render the forkedFrom config key in the editor as a non-editable field
• compiledSprout.version--> formatVersion like sprout.json
• Add a Sprout.fork(newName) and use that in the forkCurrentSprout action creator instead of the hacky / error prone approach currently.
• Add a button to fork a sprout and edit it in the webapp
• Access-Control-Allow-Origin for json and md fiels

• Streaming
• Allow Ctrl-D (I think?) to stop a streaming LLM response
• Investigate putting the prompt in the system role
• Separate out previous messages and last message in the prompt to bot.
• Include messages from the bot to the user in the history
• Summarize turns (using LLM) and replay those instead of the full message history

Set / retrieve

• Figure out an idiotmatic way to have sprouts that use the library not within the repo
• Allow running it in a way that's not node build/src/index.js
• Make sure it works if you don't provide schema (and handles it by just not saying that you can have a state object)
• Add a non-source-controlled directory for sprouts too
• Allow specifying which sprout to run from CLI
• Allow picking a sprout via CLI to run

• Plug in claude provider
• Add a converter for the stream format so computePromptStreamAnthropic and the OpenAI one can be used interchangeably.
• Resolve the bug that has existed for at least a little bit where setting the openai key leads to an infinite loop (introed in bac7ad3)
• setAPIKey -> setAPiKeys
• once anthropics/anthropic-sdk-typescript#248 is fixed, set KEY_NAMES.anthropic.com.include to true once extractStreamChunk supports it
• Make extractStreamChunk work for anthropic (and make the caller continue if content = '', whcih will happen more often now)
• Support Palm/Gemini
• Make it so the api dialog has a _firstRun (set to whether it was autoopened)
• If _firstRun, then keep the other keys behind a closed zippy
• Add a "view password" eye icon to see the key
• For each provider add a help string pointing to directions
• the zippy on conversation turn should say Advanced and print out state within
• AIProvider methods should return an object with the return value inside
• AIProvider returns resolved model in the object
• Print out the AI model in use in the advanced zippy
• Add a plugin architecture so that llm.ts doesn't have to be aware of all of the LLMs that are plugged in
• Add a promptOptions.prefererred that is the same as the shape of modelRequirements. First, clear to only required. Then look for the item that has the highest preferenceMatch score, and use that one (first one if not)
• state has a preferredModelProvider that is put into the promptOptions automatically.
• Render a red alert next to model in the zippy if it's not the default requested, and a gray one next to Advanced.
• in the api dialog allow changing the preferred provider if more than one apiKey exists.
• Two bugs: 1) if you cancel a signaller, with finsih before streamWillStart, then it will start the stream even though it shoudldn't. 2) apiKeys set one at a time lead to tickling this
• There's a weird race in SproutView where if sprout changes twice before the setTimeout is called, it will have a lastSprout that was actually never called in sproutChanged.
• Have the first run dialog only talk about openai (or have a zippy for claude / other providers
• Allow showing the API key dialog if necessary (with a little key icon button to forceOpen dialog)
• The modelForPrompt (or whatever) grows a modelProvider matching criteria. The machinery automatically sends only the models that the user has affixed an API key for
• Add a claude model to the default stack.
• Hide the API key by default in the api key dialog (maybe just expand the default provider out of a zippy
• api dialog allows adding a claude aPI key too
• If mutliple API keys, then have a dropdown to let a user pick the modelProvider to use
• Clean up the maxTokens, because it currently assumes how much context you can give but that's different.
• Document in README (and update the motivation section)
• Update node logic about setting anthropic key
• Keep track of the provider in the URL if it's not openai (the default) so people you share the URL with will try it if they dont' have a strong opinion (preferProvider)

Rationalize concept of OverlayFetcher, BrowserFetcher, NodeFetcher, etc.

Currently BrowserFetcher has a localWriteablePath that it traps out to LocalFilesystem.

Instead, create a FilesystemFetcher, (and in other cases, you might have e.g. a FirebaseFetcher).

Fetcher grows a validPath() which errors if it's a prefix this fetcher doesn't know how to deal with.

And then add a MetaFetcher, that takes a default fetcher, and then can registerHandler(pathPrefix, subFetcher) that will be called out to when it matches.

Fetcher.listSprouts() doesn't take paths, it just is supposed to list any sprouts it knows of. And the MetaFetcher calls sub ones.

Related to #19

• remove localWriteablePath
• Delete overlayFetcher
• fetcher-filestystem -> fetcher-localstorage
• Create fetcher-directoryinfo
• Remove localWriteablePath from every fetcher
• Remove overlayFetcher
• Move fetchers to be in src/fetchers
• Sprout has a metaFetcher, not a fetcher. (in app, instead of localWriteablePath, hook a Filesystem sub path)

• VScode schema validation of config.json

It's not being picked up by any services when shared for some reason

Some sprouts don't want state, they just want "previous messages".

In that case, the bot should only get a messageForUser

This will require sharing both the bot and user messages in the history, and dont' show the state in the webapp

E.g. one that wanted to allow publishing sprouts via Firestore.

Entails a clean factoring of the project.

Input and output

We're doing an ad hoc version of it, but we should formalize that as the bridge, the models are updated for it

And also use type json_object to constrain return types

Have a sub_instructions directory. The name of the .md file is the instruction.

When compiling, it generates a 10 word summary of each item and puts that in compiled.subInstructions:

{
//...
  sub_instructions: {
    "ingest_board": { 
       "summary": "<GPT-generated summary>",
       "default": true,
       "instructions": "<Full text of ingest_clues.md>",
     }
    "suggest_clue": {
      "summary": "<GPT-generated summary>",
      "instructions": <"Full text of suggest_clue.md>"
  }
//...
}

Make it so the turn can have a explain_sub_instruction with each sub-instruction type. If there's a selected sub_instruction it will be played.

config.json can have a default_sub_instruction to explain to start.

• Document pulling in sub-instructions
• npm run generate:listings should be updated in sub_instructions too
• Allow partial compilation of only the sub-instructions that have changed.
• Prompt dispatches to sub-instructions
• Allow default sub-instructions to include to start in config.json