Arbitration system for beginners, part 1

About 7 years ago, he had experience writing a terminal for the Moscow Exchange. Part of the team was fond of algorithmic trading, including me. However, I never perceived this business as a real source of income, although there were some successes in this. It is clear that to compete with banks and various foundations, with their teams of mathematicians and programmers is difficult and easier to implement in other areas.

Now at the hearing of cryptocurrency, there is a huge number of exchanges. Based on the assumption that the difference in rates on different exchanges, you can earn, I decided to explore the possibility of creating an arbitration robot. But mainly to start learning python with a real example. So let's get started.

First of all, it is required to detect currency pairs for which arbitrage trading is possible. We need couples who, as a first approximation, are actively trading, and prices on different exchanges diverge and converge.

Offhand, the work plan should be:

• Creating a database where prices of currency pairs will be stored.
• The server that will save data to the database.
• Primary analysis.

Sources are available by reference arb_analysis .

Database creation

In order to store data, you need 3 tables.

A list of stock exchanges will be stored in this table.

CREATE TABLE `exchange` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` varchar(50) NOT NULL, PRIMARY KEY (id) ) ENGINE=MyISAM; 

List of cryptocurrency pairs.

 CREATE TABLE `market` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` char(50) NOT NULL, `id_exchange` int(11) NOT NULL, PRIMARY KEY (id) ) ENGINE=MyISAM; CREATE INDEX id_exchange ON market (id_exchange); 

The table with transactions, information will also be stored here, according to data from the stock exchange glass.

 CREATE TABLE `ticker` ( `id` int(11) NOT NULL AUTO_INCREMENT, `id_market` int(5) NOT NULL, `local_time` int(9) NOT NULL, `timestamp` int(9) NOT NULL, `last` DECIMAL (11,11) NOT NULL, `low` DECIMAL (11,11), `high` DECIMAL (11,11), `bid` DECIMAL (11,11), `ask` DECIMAL (11,11), PRIMARY KEY (id) ) ENGINE=MyISAM; CREATE INDEX id_market ON ticker (id_market); 

Receiving data from exchanges

For convenient access to the exchanges there is an open source project ccxt. With the help of which it is possible in the same style to access various exchanges. However, it turned out that not everything is so rosy, information could not be obtained on a number of exchanges, and some methods did not work.

In the create_markets.py file, initialization takes place, we get a list of cryptocurrency pairs, by exchanges. This uses the load_markets () method, which returns a list of pairs for the exchange.

 name_exchange = ["acx", "binance", "bitfinex", "bitfinex2", "wex"] def create_exchange_and_market_in_db(): exchanges = {} for id in name_exchange: exchange = getattr(ccxt, id) exchanges[id] = exchange() id_exchage = db_helper.insert_exchage_to_db(exchanges[id],cnx,cursor) markets = exchanges[id].load_markets() for mark in markets: id_market = db_helper.insert_market_to_db( id_exchage, mark, cnx,cursor) 

Next, in the ex_data_saver.py file, start saving the price change for pairs:

 def save(): markets = db_helper.get_data_exch() exchanges = {} for id in name_exchange: exchange = getattr(ccxt, id) #isHas = exchange.hasFetchTickers #if isHas: exchanges[id] = exchange({ 'enableRateLimit': True, # or .enableRateLimit = True later }) cnx = db_helper.CreateConnection() cursor = cnx.cursor() loop = asyncio.get_event_loop() while True: start_time = time.time() input_coroutines = [fetch_ticker(exchanges, name) for name in exchanges] exch_tickers = loop.run_until_complete(asyncio.gather(*input_coroutines, return_exceptions=True)) count_exchange = 0 delta = time.time() - start_time for tickers in exch_tickers: if tickers is not None: count_exchange+=1 inserted_start = time.time() db_helper.insert_tick(markets,exch_tickers,cnx,cursor) inserted_time = time.time() print(count_exchange," ", delta, ' ', inserted_start - inserted_time) 

Asynchronous receiving of ticks for a specific pair is performed using the ccxt fetchTickers () method.

 async def fetch_ticker(exchanges, name): item = exchanges[name] try: ticker = await item.fetchTickers() return {name:ticker} 

Preliminary data analysis

First of all, it is interesting on which exchanges and on which pairs, the most active trading takes place, we are interested in liquid pairs. To do this, you need to calculate the number of deals with grouping by exchanges and specific pairs. As a result, we get a pair, which is an active trade.

 SELECT ex.name as exchange_name, m.name as market_name, count(*) as count_deals FROM exchange ex LEFT JOIN market m ON m.id_exchange = ex.id LEFT JOIN ticker t ON t.id_market =m.id GROUP BY ex.id, t.id_market ORDER BY m.name HAVING count_deals > 10000; 

Using SQL queries, you can find various patterns and filter out the data, but for detailed analysis, you need to create a test robot that works on data accumulated from various exchanges.

The next article will be devoted to the creation of this robot. And a test server - emulating the work of a real exchange. In the test server I want to lay down the following points:

• Latency
• Slip.
• Commission.