Sold by: Sigmodata
Deployed on AWS
Free Trial
SageMaker model package for detecting the language of a text in 20 languages
Overview
Image
Python predict
Image
POST /invocations
This SageMaker model package provides a REST API to detect the language of an input text.
It identifies 20 languages: Arabic, Bulgarian, German, Greek, English, Spanish, French, Hindi, Italian, Japanese, Dutch, Polish, Portuguese, Russian, Swahili, Thai, Turkish, Urdu, Vietnamese, and Chinese. Each prediction returns the ISO language code and a confidence score.
Use it to route incoming support messages to the right language queue, tag multilingual document sets before indexing, or filter a training corpus down to one language.
The API accepts input as JSON or CSV and supports real-time endpoints and batch transform.
We welcome your feedback at aws-support@sigmodata.com
Model and training data The model is a ModernBERT-base (answerdotai/ModernBERT-base) text classifier fine-tuned with PyTorch and Hugging Face Transformers on the public papluca/language-identification dataset. It runs in network isolation on SageMaker, so no data leaves your account.
Known limitations
- Detects 20 languages only. Text in any other language is still mapped to one of the 20, so use the confidence score to reject it.
- The model was fine-tuned on inputs up to 512 tokens. Longer text is accepted, but accuracy is only characterized to that length.
- Very short inputs (a few words) carry less signal and are less reliable than a full sentence.
- Code-switched text returns a single dominant language.
Measured performance
- Accuracy 99.8 percent, macro-F1 0.998, measured on 18,000 held-out examples from papluca/language-identification. The split is seeded (70/20/10, seed 42) and was not used in training.
- Real-time endpoint on ml.m5.large: 251 ms median per request (276 ms p95).
- Throughput on the same instance: about 24 texts per second at a batch size of 32.
Highlights
- Detect which of 20 languages a snippet is in and return an ISO code plus confidence
- Fixed 20-language set with ISO codes - ready for routing and model selection
- JSON or CSV; SageMaker real-time or batch
Details
Sold by
Categories
Delivery method
Latest version
Deployed on AWS
New
Introducing multi-product solutions
You can now purchase comprehensive solutions tailored to use cases and industries.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
Free trial
Try this product free for 5 days according to the free trial terms set by the vendor.
Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Dimension | Description | Cost/host/hour |
|---|---|---|
ml.m5.large Inference (Batch) Recommended | Model inference on the ml.m5.large instance type, batch mode | $0.10 |
ml.m5.large Inference (Real-Time) Recommended | Model inference on the ml.m5.large instance type, real-time mode | $0.10 |
ml.c5.2xlarge Inference (Batch) | Model inference on the ml.c5.2xlarge instance type, batch mode | $0.10 |
ml.c5.2xlarge Inference (Real-Time) | Model inference on the ml.c5.2xlarge instance type, real-time mode | $0.10 |
ml.c5.xlarge Inference (Batch) | Model inference on the ml.c5.xlarge instance type, batch mode | $0.10 |
ml.c5.xlarge Inference (Real-Time) | Model inference on the ml.c5.xlarge instance type, real-time mode | $0.10 |
ml.m5.2xlarge Inference (Batch) | Model inference on the ml.m5.2xlarge instance type, batch mode | $0.10 |
ml.m5.2xlarge Inference (Real-Time) | Model inference on the ml.m5.2xlarge instance type, real-time mode | $0.10 |
ml.m5.xlarge Inference (Batch) | Model inference on the ml.m5.xlarge instance type, batch mode | $0.10 |
ml.m5.xlarge Inference (Real-Time) | Model inference on the ml.m5.xlarge instance type, real-time mode | $0.10 |
Dimensions summary
You pay by the host hour for the SageMaker instance running this language classifier. Pricing splits along two choices: instance type and inference mode. Instance types include ml.m5.large, ml.m5.xlarge, ml.m5.2xlarge, ml.c5.xlarge, and ml.c5.2xlarge, so you match compute size to your workload. Each instance offers two modes: batch, for processing groups of records, and real-time, for on-demand endpoint requests. Your cost scales with the number of hours the instance runs and the size you pick. You choose one instance-and-mode combination per deployment.
Top-of-mind questions for buyers
You pay per host hour while the instance runs. Charges accrue only when the endpoint or batch job is active. Stopping or deleting the endpoint ends software charges. Underlying AWS storage or related resources may still incur their own fees, but the model itself meters running time only.
Real-time mode runs a live endpoint you send requests to on demand, billed for each hour the endpoint stays up. Batch mode processes groups of records in one job, billed for the hours that job runs. Real-time suits steady request traffic; batch suits bulk processing of stored text.
No. Cost depends only on instance type and hours run, not on how many texts you send or how many of the 20 languages you detect. You send JSON, CSV, or plain text and get a label plus confidence score. Send volume does not change the host-hour rate.
www.sigmodata.com
Helpful?
Vendor refund policy
No refunds offered but you may cancel at any time
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
Amazon SageMaker model
An Amazon SageMaker model package is a pre-trained machine learning model ready to use without additional training. Use the model package to create a model on Amazon SageMaker for real-time inference or batch processing. Amazon SageMaker is a fully managed platform for building, training, and deploying machine learning models at scale.
Deploy the model on Amazon SageMaker AI using the following options:
Version release notes
Important: security update. The inference container is rebuilt on Amazon Linux 2023, replacing a Debian base that carried HIGH and CRITICAL CVEs with no fixed version available upstream.
- The model itself is unchanged. This version is exported from the same fine-tuned checkpoint as version 1.
- Request and response formats are unchanged: JSON or CSV in, label plus confidence out.
Additional details
Inputs
- Summary
A JSON object with a "texts" array of strings to classify (or a CSV file, one text per line, for batch transform).
- Input MIME type
- application/json, text/csv
{"texts": ["Hello, how are you?"]}
Hello, how are you?
Input data descriptions
The following table describes supported input data fields for real-time inference and batch transform.
Field name | Description | Constraints | Required |
|---|---|---|---|
texts | List of text strings to classify | - | Yes |
Resources
Vendor resources
Support
Vendor support
Support contact: Email: support@sigmodata.com
Support description: contact: support@sigmodata.com
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
Cohere provides a representative AI model: Embed that translates text into numerical vectors that models can understand.
Use Cohere's multilingual embedding model to map text to a semantic vector space, positioning text with a similar meaning in close proximity
This solution categorizes different documents types like contract documents, broker submissions, invoices, etc.
This algorithm trains a supervised text classification model and uses Active Learning to provide the most relevant samples for tagging
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.