Streaming

We believe that streaming is both paramount to implementing AI generative technologies into web applications, and there should be minimal barriers to implementing streaming.

How do we make this a reality?

• First, we built an LLM-optimized schema language called Skillet
• Skillet has both streaming and partial parsing built into the core
• We make it easy – simply use the streaming keyword in your schema

---

What is Skillet?

Skillet is a Zod-like schema language that is LLM-optimized.

• Skillet is strongly typed
• Skillet purposefully limits the schema to that which is supported by LLMs
• Skillet optimizes the schema for processing by an LLM
• Skillet tightly integrates streaming

Read our docs on the Skillet schema language

---

Demo

---

Streaming Responses

Let's look at a structured completion hook in React:

import { useStructuredCompletion } from '@hashbrownai/react';
import { s } from '@hashbrownai/core';

const schema = s.object('Your response', {
  lights: s.streaming.array(
    'The lights to add to the scene',
    s.object('A join between a light and a scene', {
      lightId: s.string('the ID of the light to add'),
      brightness: s.number('the brightness of the light from 0 to 100'),
    })
  ),
});

function usePredictedLights(sceneName: string, lights: { id: string; name: string }[]) {
  return useStructuredCompletion({
    model: 'gpt-4.1',
    input: sceneName,
    system: `
      Predict the lights that will be added to the scene based on the name. For example,
      if the scene name is "Dim Bedroom Lights", suggest adding any lights that might
      be in the bedroom at a lower brightness.

      Here's the list of lights:
      ${lights.map((light) => `${light.id}: ${light.name}`).join('\n')}
    `,
    debugName: 'Predict Lights',
    schema,
  });
}

• In this example, focus on the schema specified.
• The s.streaming.array is a Skillet schema that indicates the response will be a streaming array.
• The s.object inside the array indicates that each item in the array will be an object with the specified properties.
• Note that the streaming keyword is not specified for each light object in the array. This is because our React application requires both the lightId and the brightness properties.

Here's where it gets good. Skillet will eagerly parse the chunks streamed to the output value returned by the useStructuredCompletion hook. Combining this with React's reactivity, streaming UI to your frontend is a one-line code change with hashbrown.