Sold by: Amazon
Open data
|
Deployed on AWS
This dataset provides masked sentences and multi-token phrases that were masked-out of these sentences.
We offer 3 datasets: a general purpose dataset extracted from the Wikipedia and Books corpora, and 2 additional datasets extracted from pubmed abstracts.
As for the pubmed data, please be aware that the dataset does not reflect the most current/accurate data available from NLM (it is not being updated).
For these datasets, the columns provided for each datapoint are as follows:
text- the original sentence
span- the span (phrase) which is masked out
span_lower- the lowercase version of span
range- the range in the text string which will be masked out (this is important because span might appear more than once in text)
freq- the corpus frequency of span_lower
masked_text- the masked version of text, span is replaced with [MASK]
Additinaly, we provide a small (3K) dataset with human annotations.
Overview
This dataset provides masked sentences and multi-token phrases that were masked-out of these sentences. We offer 3 datasets: a general purpose dataset extracted from the Wikipedia and Books corpora, and 2 additional datasets extracted from pubmed abstracts. As for the pubmed data, please be aware that the dataset does not reflect the most current/accurate data available from NLM (it is not being updated). For these datasets, the columns provided for each datapoint are as follows: text- the original sentence span- the span (phrase) which is masked out span_lower- the lowercase version of span range- the range in the text string which will be masked out (this is important because span might appear more than once in text) freq- the corpus frequency of span_lower masked_text- the masked version of text, span is replaced with [MASK] Additinaly, we provide a small (3K) dataset with human annotations.
Features and programs
Open Data Sponsorship Program
This dataset is part of the Open Data Sponsorship Program, an AWS program that covers the cost of storage for publicly available high-value cloud-optimized datasets.
Pricing
This is a publicly available data set. No subscription is required.
Legal
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
AWS Data Exchange (ADX)
AWS Data Exchange is a service that helps AWS easily share and manage data entitlements from other organizations at scale.
Open data resources
Available with or without an AWS account.
- How to use
- To access these resources, reference the Amazon Resource Name (ARN) using the AWS Command Line Interface (CLI). Learn more
- Description
- multi-token-completion Datasets
- Resource type
- S3 bucket
- Amazon Resource Name (ARN)
- arn:aws:s3:::multi-token-completion
- AWS region
- us-east-1
- AWS CLI access (No AWS account required)
- aws s3 ls --no-sign-request s3://multi-token-completion/
Resources
Vendor resources
Support
Contact
Managed By
How to cite
Multi Token Completion was accessed on DATE from https://registry.opendata.aws/multi-token-completion .
License
Datasets are published under CC-NC-SA-3.0 . Human evaluation is published under CC-SA-4.0 .
Similar products
TripleBlind has created the most complete and scalable solution for privacy-enhancing computation. Combining data and algorithms while preserving privacy and ensuring compliance, delivered via API.
The Kaleido Blockchain-as-a-Service is an award-winning, enterprise-grade platform making blockchain radically simple for organizations to adopt. Kaleido makes it easy to launch multi-region, multi-cloud blockchain networks on protocols including Ethereum, Hyperledger Besu, Polygon, Avalanche, Hyperledger Fabric, Quorum, and Corda. All protocol options offer support for Hyperledger FireFly, the complete stack for Web3 development.
CORE BLOCKCHAIN is a modular, secure Blockchain-as-a-Service platform for enterprise deployment of permissioned networks across cloud, hybrid, multi-cloud, and on-premise environments. Licensed per node.
KubeArmor offers robust Runtime protection for Kubernetes & other cloud workloads. Leveraging eBPF and Linux Security Modules (LSM), it enforces policy-based security measures. Safeguard your cloud native applications by limiting any suspicious behavior effectively across diversified workloads. Abstract away complexity, and embrace security.