TE Crypto Implementation, Setup and Testing

The following section is a description of the crypto gateway, herby called CG design. The overall aim with the gateway acting as an gateway to a Bitcoin and/or Ethereum network and possibly other crypto coins. The CG will hold the clients Bitcoin and/or ETH deposits that clients will be able to trade on the exchange.

Clients are able to deposit or redraw coins from the marketplace. When doing so the GC is the components providing the deposit logic and doing the interaction with the Bitcoin and Ethereum network.

By default the crypto integration is deisabled in the TE configuration i.e ./configuration/TeConfiguration.json. If you would like to try out the crypto functionality you must setup and run (or use) a Bitcoin and/or Ethereum network client. For private Bitcoin and Ethereum network testing the Bitcoin core platform and Go Ethereum, If you would like to test with public test networks. you should start to visit for Ethereum public testing and for Bitcoin public testing

But a (strong) advise is to start testing with a private network that you easily can wipeout a restart. There are a few scripts described below that will help you to get it running.

Overall Design

The CG will have one omni wallet per crypto coin that the clients have deposit to before trading crypto at the exchange. The Position Controller component is actually the component that holds the real-time position of the coins per client, the CG would just act as a depository.

The major reason for having a single omni wallet per crypto currency is;

• to minimize the tranfer fees when moving coins to/from the marketplace.
• to minimize the latency in the settlement process. When exchange participants trade with each other we would not like settle i.e transfer coins between their wallet (if there had been individual wallets) due to the fee cost and the latecy in the Bitcoin and Ethereum networks.
• Now the Position Controller will maintain the acctual position for each crypto curreny and account. When a client would like to redraw coins the amount is verified agains the Position Controller
.

Each client that would like to trade crypto currencies will have an entry in the CG (Mongo) databade. In the database the current deposit amounts will be keept together with a transaction history of all deposit and re-draw transactions.

Deposit Bitcoin, BTC

• before a client can deposit any Bitcoins the client must apply for a deposit (wallet) address. Technically that will be handed out by generate a unique address being associated with the Bitcoin omni wallet. However that does not prevent a user to send coins to the omni bitcoin wallet without having created an deposit entry. Such payment will of cause be received but be unidentified.
• a mail is sent to the client with the deposit address. An outstanding question is if a deposit address needs to be confirmed by the client? Currently a deposit enrtry gets confirmed when being created. (Who would deposit coins to someone elses account?)
• when the client deposit coins to the address being generate for client deposit the coins will show up in the Bitcoin omni wallet defined and used by the CG.
• The CG controller will identify the client by the deposit address generated. It will also update the Position Controller with the additional holdings.

Deposit Etherreum, ETH

• before a client can deposit any ETH the client must register a sourceaddress i.e. the wallet address from where the ETH will be sent. The TE Ethereum wallet will have one unique credential address. Clients will by identified by its source address. This is not optimal since any signed on client can register any address. Despite that ETH deposit addressed are added and accepted without any confirmation. Possibly this should be changed?
• The CG controller will when receiving a payment identify the client by the source address. It will also update the Position Controller with the additional holdings.

Redraw

Before a client can request a redraw of any crypto coins from the TE solution, the client must register an address to where the coins can be sent. The request must be confirmed via mail before the redrawn can take place.

When the request should be executed, the amount will be checked against the client position held by the Position Controller. This implies also consider what is being exposed in the market. I.e current position - sum of all client active outstanding sell orders.

TE Implementation

A few words about the TE Position Controller, herby a.k.a PC. The PC is used when when pre-trading is enabled. Buy orders are validated against the client cash holding, minus the current buy exposure in the market when orders are submitted. Sell orders are validate against the current holdings, minus the sell exposure in the market when being submitted.

How the pre-validation should work is likely subject for discussion and adaptation. Especially when the asset depository is not the TE System. If there is a dependency to an external depository such a broker system etc. The question then is how how/when information about the position holdings are updated between the TE system and any external deposit system. E.g are changes in the system exchanged once a day, periodically, in real-time, etc?

A simple implemenation would be to update the client holdings once every 24 hrs. The client then trade within these holdings / limits for the next 24 hrs. This however a have a significant down side. The client is trading blind considering what is happenings outside the trading system. Even if many client depo system use to work in this way it is associated with some risks. Such approach may work if the asset are just trading at one venue. How with multilisting the approach may be less optimal.

A more optimal approach is of cause if the true holder (or the collateral system) updated the TE when there is a change. In practice the bank would updated the the TE (i.e. PC) when cash was deposit / redrawn or when equities was deposit in the TE depository. And when crypto was deposit / redrwan from the TE system.

The current implementation have applied a sort of semi realtime approach;

For Equities, holdings are loaded from the file ./configuration/Positions.json every time the system is started. Any trade during the current day will applied and update the (loaded) position. It is then assumed that any external deposit change is is reflected in the position file and updated every 24 hrs or at any suiteable interval.
However the PC interface have methods for externaly updating the holdings in real-time.

Crypto will works a bit differently, it is more of a real-time approach due to the fact that we have implemented a Crypto Gateway.

• The crypto gateway component will when starting, synch with the crypto network and get the current "wallet" positions.
• The current crypto holdings are then updating the Position Controller as part of the startup.
• Any deposit of crypto asset (updateing the CG) during the day will update PC in real-time.
• Any redraw of crypto assets will be validated against the position in PC, considering active outstanding crypto selling orders. In case crypto is available the redrawn request will be accepted if not sufficient offering is available the request is rejected.

The rest is in the code. But as said the the whole pre-validation / position handling is subject for adaptation it all depends on the surrounding echo systems

Data Structures

A few words about the some of the essential data structures used for controlling the cryto flows.

AccountPosition, holds an account real-time view of the asset inventory . Updated as execution takes place and crypto assests are deposit / redrawn. At startup the crypto gateway will after synchronisation with the crypto network update the crypto position in the Position Controller. For equities the position is loaded when the system is started by read/loading the file ./configuration/Positions.json. Any trades done in any previous session for same day will update the inventory also as part of the startup.

Any deposit / redraw of crypto assets will update the AccountPosition as the happens.

Any execution taking place in the TE will update the AccountPosition as they happen.

DBCryptoDepositEntry is (Mongo) DB structure holding information about deposit / redrawn addresses being defined by clients. Clients will not benifit from deposits and cannot redraw coins unless there is a matching DBCrytoDepositEntry matching an incoming deposit transaction or redrawn request from the client.

DBCryptoDeposit holds the crypto inventory for a client/account.

Updates when TE execution takes place.

When the client deposit / redraw coins.

DBCryptoPayment used for logging incoming and outcoing crypto payments.

Setting Up Test Environmemnts

Currently private crypto network has been used for testing. The following section describes how to setup and run a private Bitcoin and Etherreum crypto network.

Bitcoin

Inorder setup and run a Bitcoin network the Bitcoin core platform is used. The core platform contains software to setup and run a one node network. You should download and install the core component

There are essentially two components in the core platform being of interest

• The bitcoin-qt application. The application is a GUI app providing a one node network and an interface for managing wallets. You have to have this application running for having a private test network.
  
• Part of the core platform is also the bitcoin-cli application. This application is required when simulating mining in the network. Via the command bitcoin-cli interface a lot of things can be done

The following steps are what is typically done to get the network up and allow the TE crypto gateway to connect to the network.

• Some script are located in the directory ./cryptogwy/bitcoin hopefully these script will make the setup easier.
• the file regtest-qt.bat (win) or regtest-qt.sh (linux) will run the bitcoin-qt application in private network mode. The script requires that executable bitcoin-qt is in the PATH.



• Run the script reqtest-qt.[sh|bat]
• In the bitcoin-qt application create three wallets: _default, USER01, USER02
• Run the script regtest-mine.[sh|bat]. with the parameter 101 It will mine 101 blocks and the mining reward will be placed in the default wallet.
• Start the TE Trading Engine server application ./bin/te.[bat|sh]
• Run the TE management GUI application ./bin/management.[bat|sh]. With this application you will be able to view wallet holdings, crypto deposit/redraw address entries and crypto payments.

You can from the bitcoin-qt application send bitcoins to a user account. The default wallet has some coins. The first thing to do is to create a deposit entry by issue a TE API rest call addBTCDepositEntry. bitcoin-qt application to the address returned in the response.

Any deposit transaction issued in the bitcoin-qt application to the TE system must be confirmed by network partcipants i.e. mined. This is accomplished by mining a block, run the script regtest-mine.[sh|bat] with the parameter 1 i.e. it will mine one block and the deposit is confirmed in the bitcoin network.

At somepoint you likely would reset everything (i.e. nuke the setup) and start all over again. There is a script ./bin/nuke.[bat|sh] that will reset the TE environment. Before you run the script you should terminate all TE application i.e the TradingEngine, the bitcoin-qt and the ETH client daemon.

The applican TeGuiTestClient that can be started via the script ./bin/guiclient.[bat|sh] provides functionality for request deposit / redraw address entries. Via the GUI it is also possible to redraw coins and monitor holdings.

Ethereum

The general approach for accessing a Ethereum networks is via a Ethereum client. The TE Ethereum component (EthereumGwy.java) is utilizing the Go Ethereum platform. The software contain two essential components for proving and controlling a private test network.

• The Geth component which basically is the Ethereum Client. The client could operate in two modes; either it runs and provedes an isolated test network or it act as a client and connects to a public Ethereum test network like Rickeby,Ropsten, etc.


• The Geth platform also contain a Javascript interface allowing Geth (javascript) commands to be executed. Some of these java commands are also available via the Geth CLI.

The script ./cryptogateway/etherreum/daemon.[bat|sh] will define and run a instanse of a GETH client that will act Ethereum client and at the same time provide a private test network. In addition the GETH client will:

• create a wallet or creditial address loaded with some coins. This will allow you to deposit ETH from this created wallet to the TE system.
• in the background run a mining process eliminating a specific need for mining Ethereum transaction.

After the GETH client has been started you should add a deposit entry allowing ETH to be deposit to an account in the TE system.

The applican TeGuiTestClient will provide a interface allowing you to performe the most essential functions for testing the Etherrum functionality. The application can be started via the script ./bin/guiclient.[bat|sh] provides functionality for request deposit / redraw address entries. Via the GUI it is also possible to redraw coins and monitor holdings.

To execute commands you shouldattach to the GETH javascript environment. This is done with the command
$ geth attach http://127.0.0.1:8545/

After attached you should load the convient methods defined in the javascript file. This is done with the javascript command
loadScript("te.js")

To find out the address of the wallet credential create in the network you should now be able to execute the javascript method;
wallet(<wallet-number>);

The script daemon will setup two wallets and the parameter wallet-number (1|2) will tell which one of the two wallets details will be shown.

Now knowing the address from where ETH will be sent you can add a Ethereum destination for a user by issue a TE API rest call addETHDepositEntry
The response will give you the address to the TE omni Ethereum wallet to which the deposit should be done.
You can allso se the the TE omni Ethereum wallet address in the TE management application under the Crypto Menu.

To deposit ETH from the wallet in the ETH client you should execute the method transfer( destination amount); in the GETH javascript environment. The destination is the address of the TE omni Ethereum wallet. The amount is the ETH being transfered. Now mining is requeried for Ethereum since that is taking place in the background.

Redraw of ether is following the same pattern. You first have to setup a destnation address by issue a TE Rest request addRedrawEntry

Note! these entries are created with the attribute confirmed false and need to be changed to true before any redraw transaction is accepted. Normally a mail should be sent out to the client as a result of the client adding a redraw entry (currently implementation missing). You can edit the entry in the Mongo database to set confirmed true. (I use Mongo Compas to browse and edit in the Mongo DB)

At somepoint you likely would reset everything (i.e. nuke the setup) and start all over again. There is a script ./bin/nuke.[bat|sh] that will reset the TE environment. Before you run the script you should terminate all TE application i.e the TradingEngine, the bitcoin-qt and the ETH client daemon.