quandl blog Menu
Tammer Kamel

Tammer Kamel

Founder of Quandl

Permalink:

Quandl Phase 2: The Democratization of Commercial Data

Today, 18 months or so since we launched, Quandl offers the world unlimited and unrestricted access to over 10 million time-series datasets. That number grows every day thanks to our small team here in Toronto and thanks to the many users around the world who contribute data to the site every day. While this is a great beginning and we are definitely proud of what we have achieved, we are nevertheless just getting started; we have aspirations way beyond 10 million datasets.

Our ultimate objective is nothing less than everything: all numerical data in the world available on Quandl. This is perhaps quite an audacious goal, but we are inspired by Wikipedia, YouTube, Google and others who have, for their respective media types, largely achieved what we seek to do with numerical data.

By “everything” we do not mean just the data that we can get for free. “Everything” means all the data that is not free too. There is a huge amount of important, valuable data that is proprietary; we are determined to offer this too on Quandl, not least because so many of our users are asking for it. Therefore, in the coming weeks we will be launching the first set of paid premium databases on the site.

Quandl’s approach to paid premium data will be unprecedented. We will not play by the rules the incumbent data oligarchy has established. Their decades-old model has not served consumers well: it keeps data prices artificially high, it cripples innovation, and it is antithetical to modern patterns of data consumption and usage. (I have discussed this before.)

Our plan is to democratize access to commercial data. Anyone will be able to buy data on Quandl. There will be no compulsory bundling, forcing you to pay for extra services you don’t need; no lock-in to expensive long-term contracts; no opaque pricing; no usage monitoring or consumption limits; no artificial scarcity or degradation. Users will be able to buy just the datasets they need, a la carte, as and when they need them. They will get their data delivered precisely the way they want, with generous free previews, minimal usage restrictions and all the advantages of the Quandl platform. And of course, the data itself will be of the highest quality: truly professional grade.

We will also democratize the supply of data. Anyone, from existing data vendors and primary data producers to individuals and entrepreneurs, will have equal access to the Quandl platform and the unmet demand of Quandl’s user base. We want to create a situation where anyone capable of curating a database can monetize their work. In time, we hope that fair competition among vendors will force prices to their economic minimum. This is the best possible way to deliver the lowest possible prices to our users.

At the same time this democratization should empower capable curators to realize the full value of their skills: If someone can build and maintain a database that commands $25 a month from 1000 people, then Quandl can be the vehicle that transforms that person from skilled analyst to successful data vendor.

Our foray into commercial data will start officially this fall with 8 pilot vendors. They range from entrepreneurially-minded analysts who are building databases to rival what the oligarchs sell for exorbitant fees, to long-established data vendors progressive enough to embrace Quandl’s modern paradigm. Other vendors will follow soon after. If you are interested in learning more, you are, as always, welcome to email me.

(Oh, and just to be absolutely clear: the addition of paid commercial data will not affect the 10 million free datasets already on Quandl in any way. All data and all features that are currently available for free on Quandl, will remain free forever. Nor will the addition of paid premium data affect our free data pipeline; we will continue to add hundreds of thousands of free datasets to Quandl every month. Our mantra remains unchanged: Any data that we can acquire for $0 will always be offered to users for $0.)

Tammer Kamel

Tammer Kamel

Founder of Quandl

Permalink:

40,000 NASDAQ OMX Global Indexes now available on Quandl

This week NASDAQ OMX became the first major index provider to distribute index data via Quandl. Quandl is now one of NASDAQ OMX’s Elite Index Data Partners, which is fantastic for Quandl users as it gives them complete and unlimited access to NASDAQ OMX’s suite of over 40,000 indexes.

The partnership is the product of shared values: NASDAQ OMX is, by far, the most progressive of the major index providers when it comes to transparency, openness, and accessibility. Quandl is built on these very principles; they are core to our value proposition. Hence the partnership opportunity made perfect sense for us.

The NASDAQ OMX index data offering is comprehensive: the iconic NASDAQ 100 index, comprehensive country and sector indexes, modern “green” indexes and specialized families like the Nordic Fixed Income indexes are all available. Quandl users, as always, get unlimited access to this data via the website, the API, Excel, R, Python, Matlab, and our other libraries.

Quandl is an open platform, so we may see other index providers on the site in the future. But the great thing about NASDAQ OMX’s suite is that it is totally comprehensive. There is almost nothing on offer from their peers that they do not produce. In fact, if you care to run some regressions, you’ll discover that NASDAQ’s indexes correlate well above 99.9% with MSCI, FTSE and others. By its sheer comprehensiveness, the NASDAQ OMX Global Indexes equip the Quandl user base with a complete benchmarking solution, which we are of course thrilled about.

You are invited to dive in to the NASDAQ OMX Indexes on Quandl.

Abraham Thomas

Abraham Thomas

I help select, add and organize data on Quandl.

Permalink:

Where Can I Find Good Historical Currency Data?

There are a number of good sources for high quality historical foreign exchange and currency data on Quandl.

A few central banks publish reference exchange rates across markets. The best of these is probably the Bank of England. BOE publishes rates to convert over 40 global currencies to the US Dollar, Pound Sterling, and the Euro. The BOE also publishes historical exchange rates for currencies that have been absorbed into the Euro zone. BOE FX rates include daily, monthly, quarterly and annual averages.

The Federal Reserve also publishes a number of foreign exchange rates as part of its weekly H.10 data release. These rates are certified by FRB-NY and published by FRB-STL. FRED FX rates include daily, monthly and annual averages for over 30 currencies against the US Dollar.

The European Central Bank publishes reference exchange rates against the Euro for over 30 currencies. In addition to these reference rates, the ECB publishes trade-weighted effective exchange rates for each currency against the EER-40. Trade-weighted exchange rates are weighted average nominal exchange rates, with weights proportional to bilateral trade volumes. This calculation method aims to capture changes in export/import competitiveness caused solely by exchange rate variations. ECB trade-weighted exchange rates come in both nominal and real (CPI-deflated) flavours.

The Central Bank of Brazil publishes monthly average exchange rates against the US dollar for about 30 major currencies. Many other central banks publish exchange rates for their own national currency against the global majors (USD, EUR, JPY, GBP).

Large international organizations like the World Bank and the United Nations all publish currency exchange rates. These have the advantage that they often include not just the nominal exchange rate, but also variations like real exchange rates, PPP-adjusted exchange rates, trade-weighted exchange rates and so on. But they have the disadvantage that they tend to be updated rather infrequently: monthly at best, annual at worst. Another interesting dataset in this category comes from the Economist magazine. The Economist publishes both nominal and PPP-adjusted exchange rates, for 60 countries against the dollar, euro, sterling, yen and yuan; the purchasing-power-parity adjustment is, famously, based on the price of a Big Mac hamburger in each country.

Of course, most bank and NGO reference rates are simply that: reference rates. They are not necessarily “trade-able” rates and may not offer either the precision or the granularity of an executable forex rate provided by a currency converter or market maker. Nor do reference rates include raw transaction statistics such as open, high, low, bid or ask. For such information, it is better to use a private sector provider. Private sector forex data providers, especially currency brokers and conversion bureaus, provide the most accurate, trustworthy and detailed foreign exchange data. We are currently working towards offering a few such sources on Quandl; if there’s any particular source you’d like to see added, please email me or mention it in the comments.

All of the above sources can be found Quandl’s API for Currency Data page. This page lists all the exchange rates available from each currency data provider, along with their corresponding Quandl codes. As with all open data on Quandl, full historical downloads are available for free, unlimited and unrestricted use, via our website, API, Excel add-in, or libraries for R, Python, Matlab and other tools.

Tammer Kamel

Tammer Kamel

Founder of Quandl

Permalink:

Quandl Open Data

Synopsis: This is, we think, the best source of historical stock price data on the internet because it is accurate, complete and 100% open.

We added a new source to the site today called Quandl Open Data. We launched it with historical daily stock price data for 500 of the largest US stocks, but we hope to get that number to 4000 this month. This new “source” on Quandl is significant for four reasons.

1 – The Data is Better

This price data is better than anything we have had before (and anything we know of elsewhere on the internet) because it includes dividends, splits and adjustments in one dataset. We calculate adjustments using the CRSP methodology, but the raw dividend and split information empowers any other adjustment methodology you may wish to employ. We update the data as quickly as we can each day.

2 – The Data is Original

Most data on Quandl is sourced from elsewhere on the internet (which we do with zealous transparency.) This new data source is different because it is “original”; the data is manufactured by us and Quandl users. The definitive version of the data actually lives on Quandl and not elsewhere. (This is a first for us.)

3 – The Data is Open

Note our terms of use for this data:

You may copy, distribute, disseminate or include the data in other products for commercial and/or noncommercial purposes. There are no restrictions whatsoever on the use of this data.

Thus we now provide the internet’s first and only totally unencumbered source of historical stock price data.

4 – It’s a Wiki

Quandl Open Data has been assigned the source code “WIKI” for good reason: this data is and will be maintained by our community. We are very excited about this project which is currently being spearheaded by us and a small set of Quandl users. We are inspired of course by Wikipedia: We want nothing less than to permanently place as much financial information as possible into the public domain with absolutely no restrictions on its use.

This project is just getting started. Our thanks to everyone who helped by contributing backfill and helping to clean. There is much more work to be done on this front. The next step is to expand coverage to more stocks. We will eventually open this process up to the entire Quandl community. In the interim it only takes an email to me to get involved right now.

  • •••
  • Thanks for leaving a comment, please keep it clean. HTML allowed is strong, code and a href.
Chris Stevens

Chris Stevens

Data curator at Quandl, and maintainer of the Python package.

Permalink:

Quandl News, November 2013

Data

Coverage

Quandl’s data coverage reached over 8 million datasets at the end of November.

Latest data sources

Latest topic pages

  • New and improved Futures topic page with 200+ contracts from 10+ exchanges. (Increased from 60 contracts and 2 exchanges). Also includes detailed commitment of traders data from the CFTC, where available.
  • New API Resources page. This page has downloadable lists of stock tickers, futures symbols, country and currency codes, and nomenclature rules for various data sources, to make using our API easier.
  • First version of our Data Request page. This page lists all the datasets / sources that our users have requested, and the current status of each. We welcome user contributions!
  • Thanks for leaving a comment, please keep it clean. HTML allowed is strong, code and a href.