Overview

Product video
Compare prompt variants and check expected responses with the Promptfoo 0.124.1 command-line evaluation workbench, packaged by Software Sushi for Ubuntu 24.04. The included example runs two prompt variants over two synthetic cases, runs four passing and four intentionally failing assertions (the failing suite keeps Promptfoo's standard non-zero exit code), and independently checks JSON results and SQLite state. The example uses a bounded local fixture and requires no model account or model download.
Launch t3.medium with at least 4 GiB RAM and 32 GiB encrypted gp3 root storage. Require IMDSv2 and keep inbound application ports closed. Connect with AWS Systems Manager Session Manager, wait for cloud-init to complete, and run sudo -iu ubuntu bash /opt/sushi/docs/demo.sh. Results are saved under ~/promptfoo-results and the evaluation database under ~/.local/share/promptfoo. The installed guide describes repeat-run readback and consistent backups. No browser viewer starts automatically. The optional upstream viewer has no built-in authentication and listens on network interfaces. The public guide shows a transient systemd unit that restricts its network access to the instance and stops it after one hour; use an authenticated instance tunnel and keep inbound application ports closed. Other instance users may access the viewer and its results.
Use your own evaluation configuration and supply model-provider credentials after launch. The optional Amazon Nova Lite example requires an instance role with bedrock:InvokeModel permission for amazon.nova-lite-v1:0 in us-east-1 and available model access. This package includes no provider subscription, model weights or enterprise features. Promptfoo is licensed under MIT; dependencies, including native components, retain their own licenses and notices delivered under /usr/share/doc/promptfoo. Software Sushi provides AMI packaging and setup guidance. This independent offering does not imply sponsorship or endorsement by Promptfoo or Amazon Web Services.
Software costs USD 0.03 per running instance-hour. EC2, EBS, data transfer and any model-provider usage are billed separately. Contact info@softwaresushi.com for installation questions. Upstream documentation: https://www.promptfoo.dev/docs/intro/ .
Getting started: https://softwaresushi-marketplace-branding-public.s3.amazonaws.com/docs/promptfoo/getting-started.html
Highlights
- Compare prompt variants with repeatable assertions and inspect JSON and SQLite results.
- Start with an included model-free example that checks passing and deliberately failing evaluations.
- Configure your own model provider after launch and retain evaluation history across restarts.
Details
Features and programs
Financing for AWS Marketplace purchases
Pricing
Free trial
Dimension | Cost/hour |
|---|---|
t3.medium Recommended | $0.03 |
Dimensions summary
Top-of-mind questions for buyers
Vendor refund policy
We offer a full refund of software charges for the first 48 hours billing starts if the product does not perform as described. To request a refund, email info@softwaresushi.com with your EC2 instance ID and a description of the issue. AWS infrastructure costs (EC2, EBS, data transfer) are not eligible for refund by us.
How can we make this page better?
Legal
Vendor terms and conditions
Content disclaimer
Delivery details
64-bit (x86) Amazon Machine Image (AMI)
Amazon Machine Image (AMI)
An AMI is a virtual image that provides the information required to launch an instance. Amazon EC2 (Elastic Compute Cloud) instances are virtual servers on which you can run your applications and workloads, offering varying combinations of CPU, memory, storage, and networking resources. You can launch as many instances from as many different AMIs as you need.
Version release notes
Packaging candidate 0.1.1; source 30b0c8ca01cf. Automatic EC2 boot, real use and host reboot verified on this exact AMI. Known High/Critical vulnerability findings remain unresolved; no security fix or security qualification is claimed.
Additional details
Usage instructions
Launch t3.medium or a larger supported instance with at least 4 GiB RAM and 24 GiB encrypted gp3 root storage. Require IMDSv2 and an instance profile with AmazonSSMManagedInstanceCore. No inbound rule or key pair is needed for Session Manager. Connect with Session Manager, then run sudo -iu ubuntu. Read /opt/sushi/docs/README.md and run bash /opt/sushi/docs/demo.sh. Keep application ports closed; use the loopback access and tunnel instructions in the installed guide. Back up your data before replacing the instance. EC2, EBS and data transfer charges apply separately.
Resources
Vendor resources
Support
Vendor support
Software Sushi provides AMI packaging and setup guidance. Contact info@softwaresushi.com for installation questions. Support coverage requires a separate agreement; this offer does not include upstream project support.
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.