When working with AI-generated content, streaming the response as it’s generated can significantly improve the user experience. Honcho provides streaming functionality in its SDKs that allows your application to display content as it’s being generated, rather than waiting for the complete response.
One of the primary use cases for streaming in Honcho is with the Dialectic endpoint. This allows you to stream the AI’s reasoning about a user in real-time.
from honcho import Honcho# Initialize client (using the default workspace)honcho = Honcho()# Create or get peersuser = honcho.peer("demo-user")assistant = honcho.peer("assistant")# Create a new sessionsession = honcho.session("demo-session")# Add peers to the sessionsession.add_peers([user, assistant])# Store some messages for context (optional)session.add_messages([ user.message("Hello, I'm testing the streaming functionality")])
import time# Basic streaming exampleresponse_stream = user.chat("What can you tell me about this user?", stream=True)for chunk in response_stream.iter_text(): print(chunk, end="", flush=True) # Print each chunk as it arrives time.sleep(0.01) # Optional delay for demonstration
import asynciofrom honcho import Honchoasync def restaurant_recommendation_chat(): # Initialize client honcho = Honcho() # Create peers user = honcho.peer("food-lover") assistant = honcho.peer("restaurant-assistant") # Create session session = honcho.session("food-preferences-session") # Add peers to session await session.add_peers([user, assistant]) # Store multiple user messages about food preferences user_messages = [ "I absolutely love spicy Thai food, especially curries with coconut milk.", "Italian cuisine is another favorite - fresh pasta and wood-fired pizza are my weakness!", "I try to eat vegetarian most of the time, but occasionally enjoy seafood.", "I can't handle overly sweet desserts, but love something with dark chocolate." ] # Add the user's messages to the session session_messages = [user.message(message) for message in user_messages] await session.add_messages(session_messages) # Print the user messages for message in user_messages: print(f"User: {message}") # Ask for restaurant recommendations based on preferences print("\nRequesting restaurant recommendations...") print("Assistant: ", end="", flush=True) full_response = "" # Stream the response using the user's peer to get recommendations response_stream = user.chat( "Based on this user's food preferences, recommend 3 restaurants they might enjoy in the Lower East Side.", stream=True, session_id=session.id ) for chunk in response_stream.iter_text(): print(chunk, end="", flush=True) full_response += chunk await asyncio.sleep(0.01) # Store the assistant's complete response await session.add_messages([ assistant.message(full_response) ])# Run the async functionif __name__ == "__main__": asyncio.run(restaurant_recommendation_chat())
Consider connection stability for mobile or unreliable networks
Implement appropriate timeouts for stream operations
Be mindful of memory usage when accumulating large responses
Use appropriate error handling for network interruptions
Streaming responses provide a more interactive and engaging user experience. By implementing streaming in your Honcho applications, you can create more responsive AI-powered features that feel natural and immediate to your users.