Crude Oil Trading and Markets - Search Queries/Keywords Dataset

 

A dataset of over 1000 scraped search keywords, related searches and search suggestions related to crude oil markets and global oil pricing.

 

• Monthly

 

 

• Crude Oil Related Keyword(s)
• Avg. Search Volume (per month)
• Keyword Difficulty (%)
• CPC (USD)
• Competitive Density (0-1)
• Results (in Million)
• Record Identifier
• Date/Time Scraped (UTC)

 

In the nineteenth and early twentieth century the global crude oil prices were relatively consistent. In the 1970s, there was a significant increase in the price of oil globally, partially in response to the 1973 and 1979 oil crises. In 1980, globally averaged prices spiked to over US$107.

Historically, there have been a number of factors affecting the global price of oil. These have included the Organization of Arab Petroleum Exporting Countries, led by Saudi Arabia, resulting in the 1973 oil crisis.

The Iran–Iraq War (1980–1988), the 1990 Invasion of Kuwait by Iraq, the 1991 Gulf War were chronologically adjacent events that accelerated price changes.

Fast forward to 2013, the oil supply glut that led to the largest oil price declines in modern history in 2014 to 2016. This 70% decline in global oil prices was one of the three biggest declines since World War II.

By 2015 the United States was the 3rd largest producer of oil moving from importer to exporter. The 2020 Russia–Saudi Arabia oil price war resulted in a 65% decline in global oil prices at the beginning of the COVID-19 pandemic.

Structural drivers affecting historical global oil prices include are "oil supply shocks, oil-market-specific demand shocks, storage demand shocks", "shocks to global economic growth", and "speculative demand for oil stocks above the ground". Source: Wikipedia, Our World In Data and the U.S. Energy Information Association 

 

We created this dataset to better understand crude oil markets and pricing. At a minimum, this dataset provided us with critical context around crude oil conversations taking place today.

There are many tools to accomplish this goal. For example, real-time search trends and keywords are easily available through API's provided by Google, Bing and third-party vendors.

As one example, you could use the Bing Suggest API for "crude oil prices" or just "crude oil". Using this API, we can have real-time formatted search suggestions returned. You can keep running similar queries and capturing results overtime to build your own dataset.

You could also use other keyword generation tools that can automatically run queries and build results into a nice list. These tools typically input keywords into Google's Suggest API, and then add letters in alphabetical (or some other) order. And if you let these tools run long enough they can help you find a lot of keywords. This is another simple approach that helps aggregate more query data.

It is important to note that these tools provide insight into what is being searched for/suggested "right now" - but we need accurate data collected over a time-frame.

To accomplish this, we built a custom wrapper around these tools that runs for a specified time duration across multiple locations tuned for crude oil related queries. The dataset that you see here on AWS is a result of this approach. Output rows can include trailing dots (..) for some or even all of the rows. You can also download a sample to see exactly how the rows appear (scroll down to the sample download section below).

Overall, there are over 900 unique terms in the dataset. Here is a quick visual of a few top terms that appear in queries in the dataset. For example,"WTI" (i.e., West Texas Intermediate) occurs in over 50 queries and "oil" in over 1100 out of a total of over 1500 queries in the dataset.

Oil ╢█████████████████████████████████████████████ Prices ╢███████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ WTI ╢█████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ Demand ╢███░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ Saudi ╢███░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ Brent ╢███░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ Storage ╢██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ Crisis ╢██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ Crude ╢██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ Russian ╢█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ Production ╢█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ Futures ╢█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ╚════════════════════════════════════════════

 

The keywords were collected (over 3 months) using a custom web search scraping framework. Results from various search engines and locales were utilized. Duplicate keywords and those that were not relevant to crude oil were filtered. Note that this dataset is a subset of a broader crude oil-related data collection effort, so if you need additional crude oil keyword datasets, feel free to reach out.

 

Crude Oil Related Keyword(s)	Avg. Search Volume (month)	Keyword Difficulty (%)	CPC (USD)	Competitive Density (0 1)	Results (M)	Record Identifier	Date/Time Scraped (UTC)
wti crude oil spot price market participants pi...	698	25	1.76	0.16	0.62M	07b0bf77 474a 4db3 badd dd731e67eede	Fri Nov 04 2022 07:24:02
crude oil sanctions significant global rece...	348	19	0.83	0.14	0.43M	82389d65 3199 4c25 aa3e ed3a7b408314	Mon Oct 03 2022 18:13:48
wti crude oil ma...	585	3	1.36	0.18	0.68M	47282527 4677 4e4d adc8 39212fc705a0	Sun Oct 16 2022 17:08:22
price of oil crud...	677	28	1.33	0.19	0.12M	d40e12b5 7aa9 468a aba0 99ff5780fe70	Fri Nov 11 2022 00:53:48
crude oil funds north p...	646	44	0.95	0.16	0.33M	dd042cf4 6db3 407c b711 2ec4a8b20363	Sun Oct 30 2022 06:23:39
physical oil storage duration...	547	30	0.64	0.16	0.06M	e2328381 aed9 48cb 8705 9a9a3125ecd0	Sat Oct 22 2022 05:38:34
russian seaborne oil level since ...	595	20	0.65	0.14	0.31M	587f5a6d ee62 46b0 8827 d2f1b1549d10	Sun Nov 27 2022 23:12:26
crude oil futures fast ...	539	8	0.94	0.15	0.61M	448dd5a2 4916 48a8 ae6e 6f6a8778fc9b	Sun Nov 27 2022 00:10:13
oil prices sea black...	509	40	1.2	0.14	0.49M	99899f16 4619 4faf 8681 a37c36280b8f	Sun Nov 06 2022 09:18:07
oil importers political professional shor...	234	8	1.35	0.18	0.62M	969a52e3 a3dc 4c7b ab7a 59eb7506bce9	Tue Oct 04 2022 10:30:11
wti crude oil r...	215	12	1.31	0.17	0.05M	826a0e9e 053c 441e 81c9 8b17f2e057d5	Sat Nov 12 2022 16:02:31
price of oil activit...	277	44	0.89	0.16	0.28M	56097afb f317 463f 95ed c9f64801c614	Tue Oct 25 2022 09:53:18
world crude oil futures short ...	180	20	1.9	0.19	0.35M	5b541cec 08b3 4f44 a382 b30db0d0b692	Tue Nov 15 2022 17:33:32
crude oil seaborne russian gove...	206	13	1.28	0.13	0.31M	af7a7d89 fb78 4009 8bb2 0a87f96b03e6	Sat Oct 15 2022 22:32:08
oil prices target ...	582	31	1.37	0.13	0.41M	525bd522 569c 417e b908 74a714b6ba22	Thu Oct 20 2022 07:04:00
producing oil drop po...	578	19	0.99	0.13	0.54M	82776d06 0e92 467f a8aa a18946cbebd0	Mon Nov 07 2022 15:56:59
saudi oil oversupply futures t...	604	8	1.83	0.12	0.42M	53094336 0aed 4a7f aa0a e1b48dda143f	Mon Nov 14 2022 04:19:13
oil company petróleos close storage c...	331	35	1.75	0.15	0.21M	cbe18d98 ea87 4c1e 90f6 7ea3b38cfbae	Tue Oct 11 2022 18:40:30
wti crude oil spot pric...	249	10	1.17	0.14	0.01M	e7cc57bc 74f7 4d19 9b6e 0058d6fdfba9	Thu Nov 24 2022 10:42:59
crude oil futures commodity m...	347	41	1.41	0.18	0.11M	8e821339 dbec 438a 8ba3 a9af47e3c612	Sat Oct 22 2022 06:18:48
brent oil standing consistent ...	377	27	1.33	0.15	0.61M	218f7cd1 5697 4d9e a7df e5134b92d07b	Sun Oct 09 2022 09:21:50
crude oil prices demand from...	487	2	1.06	0.12	0.10M	4f7ff8a6 be09 4f46 8d0e cfe3ddd48eed	Tue Oct 11 2022 23:30:07
oil production cuts funds oil p...	310	20	1.68	0.15	0.21M	4c4f6f3a 6110 4bd5 bfb4 4dc1c55e56dd	Wed Nov 23 2022 20:23:33
us crude oil in ...	739	44	0.55	0.16	0.30M	4c4d8d1b a5f4 44f2 ab01 af4d3aa3143b	Tue Oct 25 2022 11:21:14
opec oil embargo price wti...	406	43	1.49	0.17	0.19M	5db33349 7819 4bf6 aea3 7870187d6ea6	Sun Nov 27 2022 01:07:56
crude oil lowest petroleum extr...	199	15	1.02	0.14	0.44M	605de5d8 e7bb 4f89 bc53 dd67c35f13b6	Tue Oct 04 2022 05:07:29
price of oil west texas interm...	472	38	1.41	0.16	0.72M	47b306bd f249 478f b843 abb01e4edbca	Sat Oct 08 2022 20:48:18
oil liquids gulf ...	478	12	0.6	0.17	0.09M	7d00b3d5 e5cd 4825 9d6e c1355aa38d1f	Sun Nov 20 2022 08:01:47
oil prices underwent supp...	191	14	0.74	0.18	0.70M	087bd41a 46a2 4651 8b7b 2ef2346c5409	Sat Nov 05 2022 18:27:28
oil rich states come capital ...	419	14	0.75	0.14	0.22M	30c5f2b3 a118 4eba a097 e5d2fa7fa8db	Thu Nov 17 2022 05:02:38
crude oil cost product...	621	14	1.58	0.13	0.41M	0bee00fa 7277 4b55 9045 ffebbe73be8d	Sun Oct 30 2022 21:12:24
oil prices united imminent ...	499	4	0.64	0.17	0.55M	240825f4 09f2 4fd4 9cf9 42c427d5f4e7	Thu Oct 27 2022 13:12:45
oil prices e...	320	21	1.44	0.15	0.02M	28ae58fa 7a93 4ef6 9f7f 740cac75882a	Tue Nov 22 2022 07:24:24
wti benchmark crude oil main ar...	757	17	1.52	0.16	0.61M	f9e9903b 72c5 4d65 8ca5 9ee8f57188ab	Tue Nov 22 2022 22:50:28
oil price shocks december by des...	513	5	1.78	0.18	0.31M	78e5d987 fc9d 4ff1 b11c d9b02237c55f	Wed Nov 23 2022 04:24:02
producers of oil new regional oil...	560	44	1.19	0.16	0.61M	e4515b17 730e 4ab5 9721 dafad4a541b2	Wed Nov 30 2022 01:43:12
petroleum—crude oil countries exc...	244	29	1.23	0.17	0.40M	c78477bd 75f9 4d51 bbcb 1d5c78727f40	Tue Nov 15 2022 14:31:01
oil production pipe...	377	39	1.39	0.15	0.25M	0a9a8424 547d 4122 a93c d0930186ebe1	Tue Nov 01 2022 06:55:04
crude oil petroleum gas ...	528	16	1.78	0.16	0.60M	7f11e48c dd2f 4110 8eff 3b09a2379e20	Mon Oct 31 2022 00:41:55
oil producers dominant union...	470	10	1.3	0.14	0.32M	021b0ca8 f12a 4838 a0b9 bb26cd54ba62	Wed Oct 26 2022 18:40:41
oil markets due many shor...	269	6	1.91	0.16	0.47M	24fbb38d db5b 415a 86eb 4bce625d2f96	Mon Nov 21 2022 13:02:21
oil embargo last inventories comm...	250	9	1.18	0.16	0.10M	aa152d55 4668 42d0 99ce 192b866d0e20	Sun Nov 27 2022 14:14:40
oil prices funds companies ex...	458	24	0.96	0.15	0.11M	12fe7957 6cfb 4f69 abae 502f7cf23cd6	Tue Nov 08 2022 16:47:43
oil prices sachs j...	272	37	1.77	0.19	0.55M	2ff83226 20ee 45b9 af19 3391fe810fcc	Sat Oct 08 2022 17:06:28
oil prices investors institutional carbon r...	428	31	1.35	0.14	0.65M	91299581 12d6 4bee af2a 354a25d4634e	Sun Oct 02 2022 17:11:10
crude oil knock nevis s...	399	38	1.31	0.14	0.42M	5623a6d5 a489 45d1 ad1c 6efe2d4488d4	Tue Nov 29 2022 00:35:39
oil producers derivati...	423	33	1.7	0.16	0.61M	641e47ed 93f0 4bca 8b63 1ed96962b92d	Sat Oct 29 2022 18:43:12
brent oil 30 year old tight conve...	483	43	0.52	0.16	0.35M	ed684676 ff6a 4283 986a 245c2b9dfe9a	Sat Oct 08 2022 07:29:38
oil prices combination l...	458	30	1.62	0.15	0.26M	6690d328 d695 4ccf b670 3da40937e71d	Tue Nov 08 2022 21:05:19
crude oil saudi opec+ uncer...	529	31	1.07	0.12	0.14M	c5a0b1a1 504a 40eb 869b 9f8dbca8ec5f	Mon Nov 21 2022 19:51:53
oil production cuts exchange group interconti...	425	42	1.78	0.14	0.18M	a29c82dd 1991 46ab ade9 b35bdf3643d2	Sat Oct 29 2022 22:11:02
oil glut point same...	395	7	0.57	0.13	0.05M	b62d1618 a28f 4b45 8222 bda29921579a	Tue Nov 15 2022 16:13:28
crude oil book recorded h...	248	21	0.77	0.18	0.64M	a16e4857 a591 427d b127 f171c2402411	Tue Nov 08 2022 01:07:25
producers of oi...	377	25	1.79	0.16	0.23M	4b166c1d 8abd 4c94 a80a 4f50b9f6181d	Tue Nov 22 2022 10:34:48
russia saudi arabia oil price war prices trade...	439	33	0.72	0.18	0.54M	96a70458 6f69 456c a87a b3e02e50b951	Sun Oct 09 2022 01:41:31
price of oil supplies energy ...	595	31	1.45	0.13	0.44M	a09924b4 17bf 4b30 8f8e 21666573620d	Sat Oct 22 2022 14:12:09
u.s. oil production financial demand dr...	436	16	0.61	0.14	0.66M	aa35ba0d 35be 4d7d ba81 aca7513271cb	Mon Oct 17 2022 18:13:16
aramco oil facility trends strong ...	498	2	1.2	0.18	0.54M	c87380ee fa4a 4300 aba4 8706195b0ec7	Sun Oct 30 2022 17:24:33

 

• Save time finding actionable crude oil trading and pricing related content. For example, consider running the queries in the dataset and reviewing the results. Or you could use the queries to better understand crude oil markets.
• Fine tune crude oil trading strategy.
• Have a professional 360 degree view of crude oil related queries.

 

It's easy to get setup on AWS and takes no more than a few minutes.

• Step 1 - Go to aws.amazon.com and click "Sign in to the Console"
• Step 2 - On the Sign In page click "Create a New AWS Account"
• Step 3- Fill in your email and payment details.

Here are some tips for Step 3:

• Tip 1 - You will have the option to select between a "Personal" or "Professional" AWS account. The difference between these options is the information that AWS will ask you for. Both account types have the same features and functions. For example, if you are buying on behalf of a company and using a corporate credit card, consider picking Professional. Otherwise, consider picking Personal as your account type.
• Tip 2 - You will also have the option of picking a support plan. Even if you go with Basic or free option initially, you can always upgrade your AWS support plan anytime.

After you complete Steps 1-3, a confirmation page appears that indicates that your account is being activated. Check your email and spam folder for an email message that confirms your account was activated. Activation usually takes a few minutes but can sometimes take up to 24 hours.

During activation, you can sign in to your new AWS account, but you might see a Complete Sign Up button. You can ignore the button. After you receive the activation message, you have full access to all AWS services.

(https://docs.aws.amazon.com/accounts/latest/reference/welcome-first-time-user.html )

• Step 4 - At this point you should have an activated AWS account. You can now go ahead and subscribe to any AWS Data Exchange dataset. You can accomplish this by following the instructions at:

  (https://docs.aws.amazon.com/data-exchange/latest/userguide/subscribing-to-product.html#subscribe-s3-data-access-product )

 

---

 

There is no other place where customers can find data files, data tables, and data APIs from a vast portfolio of third-party datasets. AWS continuously innovates to make the world's third-party data easy to find in one data catalog, simple to subscribe to with consistent pricing options, and seamless to use with AWS data and analytics and machine learning services.

 

---

 

If you have questions about this crude oil keywords dataset or other related datasets on AWS, contact us at awsdataexchange@refdb.org