Overview
This product can help to stress the inference server with concurrent queries with custom large data and analyse the server resource utilization (e.g. GPU utilization, GPU memory, CPU utilization and CPU memory) against one of multiple GPUs. Monthly charge is for support and customization on the go.
Highlights
- This product can help to determine and analyse the large data
- You can input any json based data url. Server is able to ingest data and using those data you can chat anything with those data
- Prior support provide on mail and customization on the go
Details
Typical total price
$1.556/hour
Features and programs
Financing for AWS Marketplace purchases
Pricing
- $850.00/month
Instance type | Product cost/hour | EC2 cost/hour | Total/hour |
|---|---|---|---|
g4dn.xlarge | $0.55 | $0.526 | $1.076 |
g4dn.2xlarge | $0.55 | $0.752 | $1.302 |
g4dn.4xlarge | $0.55 | $1.204 | $1.754 |
g4dn.8xlarge | $0.55 | $2.176 | $2.726 |
g4dn.12xlarge | $0.55 | $3.912 | $4.462 |
g4dn.16xlarge | $0.55 | $4.352 | $4.902 |
g4dn.metal | $0.55 | $7.824 | $8.374 |
g5.xlarge Recommended | $0.55 | $1.006 | $1.556 |
g5.2xlarge | $0.55 | $1.212 | $1.762 |
g5.4xlarge | $0.55 | $1.624 | $2.174 |
Additional AWS infrastructure costs
Type | Cost |
|---|---|
EBS General Purpose SSD (gp2) volumes | $0.10/per GB/month of provisioned storage |
Vendor refund policy
No refund policy
Legal
Vendor terms and conditions
Content disclaimer
Delivery details
64-bit (x86) Amazon Machine Image (AMI)
Amazon Machine Image (AMI)
An AMI is a virtual image that provides the information required to launch an instance. Amazon EC2 (Elastic Compute Cloud) instances are virtual servers on which you can run your applications and workloads, offering varying combinations of CPU, memory, storage, and networking resources. You can launch as many instances from as many different AMIs as you need.
Version release notes
Release notes:
- Enable the Llama3 model family
- Support for AMD-based GPU handling in standalone installations
- General bug fixes
Additional details
Usage instructions
Once the instance is run, all the services will come up automatically once instance boots up.
It is suggested to manually configure your Security Group/Firewall settings to access your instance. Currently, the 1-Click Security Group opens only to port 22 and 80 so that you can access your instance via SSH using login 'ubuntu' username. If you chose the 1-Click Security Group, you can change it later to enable other applications to use the AWS Console or API.
To connect to this instance via ssh, you must have a new keypair generated or use an already present keypair pem at the time of launching the EC2 instance. Using this keypair, you can login to the instance as below: ssh -i <key-pair> ubuntu@<public_IP_of_the_instance>.
Alternately,you can also login to the instance via the AWS console by clicking on the"Connect Instance" option on the EC2 instances page. The username must be "ubuntu".
The way to the check the logs are
- Connect to the instance via ssh or console.
- Type "sudo su" to enter super user [No password required]
- type "bv-ai-stress verbose" and hit enter. You would be able to see the logs as below. More logs would generated once the inference server is being used.
root@ip-<>:~$ bv-ai-stress verbose Streaming logs for process: bv_inference_stress (ID: 0) [bv_inference_stress] 2024-09-06 09:07:12 - INFO - 127.0.0.1:35034 - GET / HTTP/1.1 - 200 OK
[bv_inference_stress] 2024-09-06 09:07:14 - INFO - 127.0.0.1:35036 - GET /models/ HTTP/1.1 - 200 OK
[bv_inference_stress] 2024-09-06 09:07:14 - INFO - 127.0.0.1:37152 - GET /favicon.ico HTTP/1.1 - 404 Not Found
[bv_inference_stress] 2024-09-06 09:07:17 - INFO - 127.0.0.1:37166 - POST /queue-count/ HTTP/1.1 - 200 OK
Note: Right after launching the instance, it may take a few minutes to show the logs. Try after sometime if you do not see any logs. Two option to access platform.
- Using UI we can access the platform. we can access UI via http://<instance-public-ip>.
- Using cli we can access the platform. We can see help/docs section via this command "bv-ai-stress -h".
For any issues or queries, contact our support team (at aws-mp-support@bhojr.com ). For more information on the product, check out the link: https://www.bhojr.com/prod/ai-inference-stresser.html
Resources
Vendor resources
Support
Vendor support
Support can be available on mail address aws-mp-support@bhojr.com
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
