Mike Fishy and Automation System Programming (Bots)


#22

Sounds interesting. What algorithms do you use to determine when to make a trade (sell & buy signals)? This is mostly big data analysis, right?


#23

Hound is the process which calculates the probability of a good entrance.

At the moment it is using K-nn and some Logistic regression.
Today I have been coding in the Weibull distribution algorithm.

Also, Hound uses mostly small datasets, usually 40 mins worth which is all that is really needed to go after the 1.x% to 3.x% gains (or 0.1x% to 0.2x% loss).

It is easy to see variances too, by checking the book orders (since the next order will either fill a bid or an ask order in the order book - unless there is shenanigans, which does happen).
Predictive mathematics doesn’t always work too well when people are involved, hence the stop loss functions.

On average though, it does surprisingly well considering this type of market, it’s certainly been a good challenge for an old Fish and I have enjoyed the ride, but wait… the ride ain’t done yet!

On a side note, for the HODLers, I have gone bigly in Nintendo shares and have been since last year.
The shares are currently 2x from April last year. If you do as well, you might thank me later, they will most probably be the clear winners of VR and AR, as they have five very significant advantages, which you can research if you wish.

New patents filed by Nintendo have surfaced that suggest virtual reality functionality…

Not Financial advise, be Fishy on your own. It is good to diversify your investments though.

Virtual Fishy


#24

Fishy bots have been fishy this week and most my time was spent ironing out the bugs.

For people that code, you know how it is. You get to 99% done and it stays at 99% forever LOL
(just like those pron downloads we pay for in Verge…)

Stay Fishy


#25

As someone who founded BotList and has been in the bot industry for close to 3 years, this is next level!

I need to bookmark this thread and am looking forward to future updates. :star_struck:

PS: Huge Transformers fan here… Gamer tag is Just Prime :optimusprime:


#26

Woah Mikey he’ll eat anything. Awesome thread super generous and maybe narcissistic :wink:

Definitely gonna watch this thread man love them bots brah…thx again for sharing feel like my IQ just went up a few points.


#27

When you code for the love of coding and build something you use yourself that is successful, then YUP!

I have always loved coding, been doing it for 36+ years and I am constantly on the lookout for new projects to jump into. However, I have had a lot of experience in building bots and automation, especially in the B2B arena, so through the “school of hard knocks”, I have learned how to build systems that are robust.

The bots are averaging 472 trades per day and are working as expected to the most part. I still have an issue around partially filled orders that I am working on, which is what the Weibull algorithm was for. Weibull is mostly used to predict the amount of rainfall when it is going to rain, so I am using it to attempt to predict the volume of the next trade cycle in effort to ensure more orders get filled rather than partially filled.

I also added a new feature that allows me to add trading pair exceptions on the fly rather than needing to restart any of the processes. Sometimes you just don’t want the bots to trade specific pairs that you might not like due to news releases.

There is still work to be done, but the system is now at a stage that I would call “Alpha release”.

In the future, I may release a Windows version and the source code, though if you wanted to look at an open source project that is going well now, the Gekko project is one to look at:

Stay Fishy


#28

Very cool man you just gave me homework :wink:


#29

Nice thread, @Mike_Fishy. With all the content floating on The Pub, I completely missed it until today.

I’ll have to do a bit of write up myself on the bots Ive been building. One thing’s for sure, as you translate what you do yourself as a human into code, the emotions of the trade and the fandom-attachment for a coin completely begins to fade away and the bots start picking and trading stuff you’d never look twice at otherwise.

Plus, teaching the bots to trade +1%, ~ +3% while stopping losses consistently at -0.25% (my thresholds) makes for a winning combo overall as long s as you’re entering profitable trades around 60% of the time.

I’m still working on the screening/filtering to pick the best entries but it’s all simple rules based and not yet touching AI.

It’s interesting to see MongoDB being used here. I settled on MySQL and Redis combo to make things fly and I separate the bots into dockerized containers to maximize CPU utilization since I’ve built everything with Ruby and Javascript. Like you, I have one that collects the data, one that sorts it and signals for the trade entries. Then there’s the actual trading bots themselves. Most of my energy is focused here because as I build the trading strategies, it’s easy for me to manually enter trades ideal for each trading strategy. The problem is doing it for more than a handful of CDAs because I cannot possibly watch more than 15 ~ 20 diligently.

And fun choices on the bot names. I’m a little more generic for the most part, but my favorite one so far is the trio of bots that work together, called Snap, Crackle, Pop – yep, after the cereal. Snap’s job is to secure breakeven on the trade then pass the buck to Crackle. Crackle’s job is to earn at least 1.5% then sell on next peak and hand off to Pop. Pop then computes Chandelier’s exit as means of chasing to maximum profit potential on a position (which can go on for a few days). Snap usually gets breakeven in a few minutes and trades enough to lock that profit down. Crackle can take a few minutes to a few hours to achieve it’s 1.5% lock on profits.

Anyway, enough for now!


#30

Nice, I like the snap, crackle, pop, good choices :smiley:

I have been crazy busy at coding in Sideswipe

This one works on the 24 hour statistics to buy in on the daily dips and then hodl.

Still much work to be done, bugs to be fixed, and automated recovery procedures to be written.
Ultimately, the whole system should be stand alone and not require attention, which is a bit of a pipe dream since computers are so unreliable, but if it is stable enough and able to recover itself from the majority of issues, then I will be happy. My Forex bots require almost no attention anymore, so this project is keeping this old Fishy busy and giving me purpose in life :slight_smile:

Stay Fishy


#31

Speaking on reliability, I’m almost positive much of the grief comes from regular waves of DDoS attacks on the exchanges.

They don’t talk about it, but when you’re interacting on API instead of website frontend, you can definitely get a sense of when the exchanges are either under heavy loads they’re not prepared to handle or they’re steadily under DDoS attack.

Binance seems to have done the best job of mitigating DDoS as well as having ample resources behind driving their services. Bittrex suffers timeouts far too often, esp. of late. Kucoin, Trade Satoshi, Cryptopia all seem to get hammered time to time, often outages on their APIs lasting for days at a time.

Speaking of Bittrex, they have at least a version 3 of their API, but only document up to version 1.1. Have you found any documentation on their 3.0 API anywhere other than digging through other people’s API wrappers? I’m basically chasing a market order call as I have to simulate market orders by targeting specific order book asks/bids with limit orders until desired amount purchased/sold and it’s annoying when compounded with Bittrex often timing out on the individual API calls.


#32

speaking to bugs…do you have automated testing and ability to replay/backtest a sequence of trades? I recently put a lot of time into being able to capture the trades gone wrong so I can fix bugs or develop entirely new strategies against. I also wanted to be able to replay not only the existing strategies but new strategies against all prior failed scenarios to ensure no regression introduced. Over time, I also had the idea that if I capture and resolve enough such scenarios, I’ll also eventually have the training dataset for when I introduce AI into the picture.


#33

If you get a bot working in Win I’d be interested in learning more. Although my programming skills are better that “Hello World” I’m no where near your level and doubt I ever will get passed the “know just enough to really screw things up” stage, but I like a challenge so I press on. I’m still asking the age old question “To bot or not to bot” but switching internet provider next month and if stable then might try a bot on one exchange see how it goes. Now when people mention 6 BTC and 20 BTC I’m like wow must be nice! So is there a certain threshold where bots perform better, more reliably or profitable? I know the more you have to more options will be available just wondering if there was a certain point where any lower and it’s just not worth using a bot.


#34

This is one of the coolest threads on the pub


#35

@mwlang Yes, DoS attacks on the API do happen, DDoS attacks are more rare. It’s easier and takes less resources to hammer the API with legitimate calls, but at least Binance has some rate limiting protection. Also since they are in AWS, the advanced DDoS mitigation you can buy from AWS is very good and worth the extra.

I also noticed even Binance change the API (and trading pair decimal place rules) without updating any documentation. This is a common theme these days on almost all programming I see and is a side effect of Agile methodologies where sprints don’t take into account updating the surrounding media, be it documentation, example code, help forums, FAQ’s and so forth (even the ability to roll back from a change is often skipped, to keep pushing out more and more updates faster). It’s not the fault of the methodology.

Also, no I do not do any back testing, though I do perform a lot of analysis and limit risk by limiting trade volumes until I am satisfied it is working as intended.

@Fatninja I do plan on doing something on Windows later. There is no real threshold for a bot, even small trades are profitable when you do hundreds per day. I actually had to tone it back a bit, it was executing up to 800+ trades a day at one point until I added in a feature to slow it down and make more careful entries. The number of stop loss exits dropped dramatically and became much better at it’s job. I have certainly learned a lot as compared to Forex trading. The addition of the Weibull algorithm has helped a lot, though I subtract 5% from it’s predicted volume to allow for more trades on different pairs to happen at the same time (spreads risk a bit better).

When I start building a Windows one, I shall post progress here. Linux is just superior and faster to do these things on and get all the rules/strategy right.

Stay Fishy


#36

Anyone have advice on Gekko strategies? Just messing around a bit but so far the included ones are really bad and I am having issues trying to build one.


#37

This is true, but unlike the other exchanges, they actually provide an API call with the details so you can stay on top of if programatically. I fetch this info from https://api.binance.com/api/v1/exchangeInfo every four hours and it keeps me out of trouble there.


#38

Oh! Didn’t notice that one, thanks for the tip :smiley:

Cheers…

Mike Fishy


#39

I love this thread!

I don’t know anything about trading bots or programming but after read this I have more understanding.

Is play around with Gekko the best way for me to learn more?

Any pointers are appreciated


#40

Hi Team,

I completed the code of Sideswipe and integrated into Hound.
Then adjusted the volume to use the higher prediction of the Weibull algorithm.

This allows the bot to trade with higher volumes on less trades per hour.
It has also allowed Hound to calculate the most probable trades based on an
extrapolated K-NN algorithm together with the Sentiment value resulting in
less stop loss exits.

From midnight to 7am today, the current running percentage is 62.14%
So I am quite happy with the results and consider my code to be mostly done and stable.

However, today I would like to discuss a conversation I had last night over several
shots of Saki with a good friend of mine from Japan (after which we went to the VIP
Karoke and strangled some cats - lolz)

Are bots good or bad to the markets?
Can you automate the trading process?
(And more importantly, Should you?)

Cryptocurrency trading is a source of income for a few people in the sphere.
Very few of these traders actually have the time to stare at the price charts
the whole day. This is where trading bots become really useful. They’re
basically tools that can do these redundant, but important activities like
price monitoring, stop loss, alerting etc.

This way, trading can be done even when you are sleeping, out drinking Saki or
singing your heart out at Karoke. Besides, it is no secret that bots are faster and
more effective than humans when it comes to functional things and working with numbers.

Bots can save you time but don’t be completely blind and expect the bot to
keep on making money for you. Bots aren’t a fire-and-forget technology. Keep your
eyes open and do some manual work like checking whether the bot is making right
decisions or not. A mixture of manual and automated decisions is really required
and bots require adjustment to suit specific market trends.

A few bots and tools available for crypto trading are:

  1. Gekko
  2. GUNBOT
  3. ZENBOT
  4. C.A.T.
  5. Tribeca
  6. Butter Bot
  7. TA-lib
  8. Crackin Kraken
  9. Cryptohopper
  10. HaasBot
  11. Tradewave
  12. BTCRobot

However, learning to code your own using strategies from these bots, will generally
produce results that you will be more comfortable with.

Will your own bots be more profitable? Maybe or maybe not, but since it was your own
work that went into the bot, you will be less dissapointed with any mistakes or losses.
Plus you will have the ability to correct these errors, rather than becomming
frustrated over the bot making losses and causing you grief.

A gamer keyboard has the WASD keys all worn out, an artist will have the Z key all
worn out, while a keyboard on the ground in a million peices, belongs to a bot programmer.

So, why are bots good or bad?

Common arguments for why bots are bad include:

  1. Bots can restrict market growth by constant buy and sell
  2. Bots run by large volume traders can manipulate the price
  3. Manual trading can have benefits that surpass bots
  4. Bots can loose all your money without even caring
  5. Bots do not understand risk tollerance without constant attention
  6. Bots hurt manual traders by sniping trades faster
  7. Bots can be manipulated and taken advantage of

Common arguments for why bots are good include:

  1. Bots provide liquidity and a heartbeat to the market
  2. Bots help traders as a tool for capturing more trades
  3. Bots can help stabilise the market and reduce massive falls
  4. Bots can be programmed to execute any strategy you wish
  5. Bots provide YOU with more freedom by doing tasks for you
  6. Bots can perform hundreds of trades increasing market volume traded
  7. Bots can take advantage of market inefficiencies

In closing, Bots have good and bad ramifications to the market.

Once most bots pick a direction, they stick with it until the next textbook support
or resistance level. Humans are predictably unpredictable, thus can cause massive
shifts in trading without textbook strategies. The biggest issue with bots is
when humans become involved. If there is a sudden sell off by a human, all the bots
kick in and sell off as well using the textbook stop loss strategy. This in turn
can cause a massive market down turn. However, the same is true in the opposite
direction, as bots buy and sell the trend and is how they are manipulated by
the larger volume traders.

The smart traders monitor the charts, trade in a vacuum, and follow the bots.

Stay Fishy


#41

In the unlikely event that we ever meet, I would like to buy you a beer or two or three…or maybe just send you some XVG. These posts are rich (XVG lol)

Are bots good or bad to the markets?
Can you automate the trading process?
(And more importantly, Should you?)

I’ll add some substance
(Here)