📦 v0.0.3: Streaming Chat

[roma-glushko]

Summary

✨ Bringing support for streaming chat in Glide (integrated with OpenAI, Azure OpenAI and Cohere)
✨ Started handling 401 errors to mark models as premaritally unavailable (e.g. when API key was not correct)
🐛 Fixing the panic related to swagger.yaml file
🐛 Fixing Anthropic chat workflow by passing API key correctly
🔧 Improved Cohere param config and validation

Changelog

Added

• ✨Streaming Chat Workflow [Lang Chat] Implement a PoC of a async router with streaming lang API #149 🔧 [Stream Router] Extend the model health & latency tracking functionality to account for streaming API #163 🔧 [Lang API] Track what if the chat chunk is the final one #161 (@roma-glushko)
• ✨Streaming Support for Azure OpenAI [Lang] Integrate Azure OpenAI with Chat Streaming #173 (@mkrueger12)
• ✨Cohere Streaming Chat Support [Lang] Integrate Cohere with Chat Streaming #171 (@mkrueger12)
• ✨Start counting token usage in Anthropic Chat 🐛 Anthropic: Fix Wrong API key header & start counting usage tokens #183 (@roma-glushko)
• ✨Handle unauthorized error in health tracker 🔧 [Resiliency] Handle 401 error in HealthTracker #170 (@roma-glushko)
• 🔧 🔧 Set router context for stream response chunks #195 🔧 Signal end of chat stream in case of error #196: Set router ctx in stream chunks & handle end of stream in case of some errors (@roma-glushko)
• 🐛🔧 [Lang] Handle generation is too long as a finish reason #197: Handle max_tokens & content_filtered finish reasons across OpenAI, Azure and Cohere (@roma-glushko)

Changed

• 🔧 💥 🔧 [Lang] Cohere: Expose most chat params #198: Expose more Cohere params & fixing validation of provider params in config (breaking change) (@roma-glushko)
• 🔧 🔧 Render Durations in a human-friendly way #186: Rendering Durations in a human-friendly way (@roma-glushko)

Fixed

• 🐛 Fix Anthropic API key header 🐛 Anthropic: Fix Wrong API key header & start counting usage tokens #183 (@roma-glushko)
• 🐛 🐛 [Bug]: panic: ./docs/swagger.yaml file does not exist #209: Embed Swagger specs into binary to fix panics caused by missing swagger.yaml file (@roma-glushko)
• 🐛 🐛 [Lang] Cohere: Streaming Workflow is not working #200: Implemented a custom json per line stream reader to read Cohere chat streams correctly (@roma-glushko)

Security

• 🔓 Update crypto lib, golang, fiber 🔒 Upgrade the crypto lib (CVE-2023-48795) #148 (@roma-glushko)

Miscellaneous

• 🐛 Update README.md to fix helm chart location 🐛 Update README.md to fix helm chart location #167 (@arjunnair22)
• 🔧 Updated .go-version (@roma-glushko)
• ✅ Covered the telemetry by tests Improve Code Test Coverage #146 (@roma-glushko)
• 📝 Separate and list all supported capabilities per provider 📝 Separate and list all supported capabilities per provider #190 (@roma-glushko)

---

This discussion was created from the release 📦 v0.0.3: Streaming Chat.