At Superflows, our mission is to give everyone access to an expert in the software they're using.

While building the Alpha version, I was struck at how often I had to hold myself back in not building entire new massive sections of the product.

We ruthlessly prioritised the core features that would allow us to test our core hypothesis: that we could build a useful developer tool to help with building an AI assistant.

This page was born out of this process. Exploding with ideas, I wrote them down mostly so I didn't forget them all. It's also not quite a roadmap since it's not ordered yet - the order will depend on user feedback.

I've split them into 4 sections, based on the 4 main areas of the product:

Core Product​

1. Action chaining: allow the LLM to output multiple API calls in 1 LLM response where later API calls use the output of previous calls.
2. Scale to large API specifications: GPT's performance drops considerably when the number of API endpoints to select from grows. We have attempted several potential solutions, but none have worked satisfactorily so far. We believe a combination of intelligently selecting the most relevant endpoints and using fine-tuned models could be the solution.
3. Enhance the OpenAPI schema provided
   1. This could be as simple as adding descriptions to parameters or endpoints that lack them
   2. Or updating the schema based on errors received from calling the API where there are mistakes
   3. Also, updating it based on user feedback on whether specific responses were helpful or not in answering their queries
4. Add capability to answer questions based on the software's docs (standard question answering approach: scrape, chunk, embed in vector DB)
   1. E.g. How do I contact support?
   2. Can I speak to my account manager?
   3. We're at our usage limit, how do we upgrade?
5. Enable the LLM to select non-HTTP actions:
   1. Calling functions in the frontend which are exposed by developers
   2. Enable redirects to other pages in the frontend
6. Help the AI better understand how to use the API by providing relevant sections of the software's docs
   1. E.g. to create a report, you first must set up a segment and then add the report to that segment

Large Language Model​

As an Open Source company, we are really excited to getting to grips with Llama 2.

We hope we can make meaningful contributions in the coming months to the Open Source community that will form around it.

We think it offers massive potential for improving the performance of our AI assistant.

1. Improve the base model (fine-tune & LoRA on Llama 2)
   1. The hope: Make the model faster, cheaper, more accurate and able to be deployed on your own cloud infrastructure
2. Customer-specific fine-tuning:
   1. Enable customers to fine-tune the model on their own data

Dashboard​

1. Persist chat history: enable developers to see the chat history in the playground
2. AI assistant evaluation: enable developers to evaluate the model's ability to answer questions more thoroughly before deployment
   1. Generate potential user questions
   2. Evaluate the model's ability to answer those questions on a test database configuration
3. AI Assistant version control
   1. Add development versions - so changes made on the dashboard don't affect production until that branch is set to be the main branch
4. Analytics: enable developers to see how their AI assistant is being used at scale
   1. Add analytics section to the dashboard
   2. What questions are commonly asked?
   3. Integrations with other tools could be useful e.g. Segment
5. Feedback: enable users to give feedback on the answers they receive
   1. What kinds of questions are not answered well?
   2. What are the most common questions?

Frontend​

1. Frontend UI components - add new views other than the sidebar
   1. Add single page view
   2. Add command bar view
   3. Enable developers to add their own visual components (e.g. visualising a graph)
2. Make frontend components fully stylable through setting CSS
3. Component libraries for
   1. Svelte
   2. Angular
   3. Vue
   4. PHP ...

Tell me what you think - I added comments below (if it doesn't show, click here and scroll to the bottom).