Overview
FloTorch is an open-source, free-to-use tool that enables users to efficiently prototype and optimize Retrieval-Augmented Generation (RAG) workloads. It provides a robust framework for:
- experimentation and performance tuning,
- allowing users to configure critical hyperparameters,
- run experiments securely within their AWS infrastructure, and
- maintain full data sovereignty.
Highlights
- Optimized Experimentation: Tunes hyperparameters and tests strategies for cost, latency, and performance optimization. Saves weeks/months of valuable time.
- LLM and Database Integration: Enhances embedding models and vector databases for improved retrieval efficiency.
- Context Management and Dashboards: Fine-tunes LLM prompts and provides dashboards for data-driven decision-making.
Details
Features and programs
Financing for AWS Marketplace purchases
Pricing
Vendor refund policy
Not applicable as the product is free
Legal
Vendor terms and conditions
Content disclaimer
Delivery details
FloTorch Container Based Installation
- Amazon ECS
Container image
Containers are lightweight, portable execution environments that wrap server application software in a filesystem that includes everything it needs to run. Container applications run on supported container runtimes and orchestration services, such as Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS). Both eliminate the need for you to install and operate your own container orchestration software by managing and scheduling containers on a scalable cluster of virtual machines.
Version release notes
Release notes for v1.0.2 available here: https://github.com/FissionAI/FloTorch/blob/main/release_notes/Release%20v1.0.2.md
Additional details
Usage instructions
All the installation steps for FloTorch are outlined here:
https://flotorch-public.s3.us-east-1.amazonaws.com/installation.md
Resources
Vendor resources
Support
Vendor support
Primary Support Methods:
- GitHub Issues: a. Purpose: Bug Reports, feature requests, and discussions b. Response Time: Best-effort basis by community maintainers.
- GitHub Discussions: a. Purpose: Technical questions and community help b. Format: Q&A, troubleshooting, best practices
- Documentation: a. Location: Project Wiki and README b. Content: Setup guide, best practices, troubleshooting.
Reach out to discuss AI transformation and implementation needs: email: info@flotorch.ai
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.