Behind the Scenes at a CS:GO Gambling Site

I recently had the pleasure of working as a back-end developer for a CS:GO gambling site. These controversial sites allowed Counterstrike: Global Offensive players to trade and gamble “skins“, which are basically in game models and art.  Each skin has a rarity and real value, like old-school trading cards. The value is based on a game company called Valve who issues the skins and determines rarity and an open market of players who buy and sell them.  All the sites like ours relied on Steam; Valves online gaming platform and store. The market for these type of sites grew large under Valves hands off approach (some estimating a billion dollar industry), and dozens of gambling sites popped up, we were one of them.  Eventually there was too much negative press surrounding these sites and Valve felt it necessary to shut the sites down. We closed our site on July 19th 2016.

The Beginning

I had been contacted by a friend to help him work on the back-end for a CSGO gambling site, so I was able to get involved with the development of the site from the ground up. I personally had never heard of these types of sites and knew nothing about the market for skins. It sounded like a very interesting and challenging project.  A lot of big numbers were thrown around, so I said sure, i’ll help.

My role grew to what was eventually become the games master.  I wrote, optimized and maintained the flagship games on the site.  The CSGO gambling market was already pretty saturated at this point, with dozens of sites out, but our team was confident this would be a success. I had no idea, i just took their word for it. Our first software commit was April 3, 2016.

The Team

The entire core team was 5 people. 1 front end developer, 2 back-end developers , 1 marketing, and 1 sales.  I would consider myself sort of a minor member on the team, since I was brought in second hand, but my contributions were significant enough to warrant a small percentage of the revenues.

The front end, marketing and sales people had previous experience creating their own CS:GO gambling site. They sold it – but their back-end guys stayed with the old site.  They wanted to start a new invite-only site, this is why my friend and I were brought on. Eventually we would bring on several site administrators and moderators to help manage the community, but for the most part the team was small and everyone had their critical roles.

The Site

Our site differed from most of our competitors because we were invite only. You had to know someone who was a member to get in.  Sort of a gimmicky thing, but it seemed to work. It made us feel a little more exclusive to players and we offered rewards for members who referred their friends.

We also had many games, while most other sites focused on 1 or two. We added several experimental games as well.  We tried to be very interactive with our players, we had an email support system which allowed players to contact us with problems. We had site moderators and admins who managed the chat and reported any issues as well. We did weekly surveys to get feedback about the site for improvement. I think many of these gambling sites get a bad rap for being scammy, but the “culture” of our team was to do what we could to resolve players issues and take feedback to improve the site for them.

Most of our popular games were copies of what was already out there.  We launched with 2 games, one we called “Bust”, a clone of BustaBit – originally a bitcoin based gambling game.  The other was a simple coinflip game which pit two players against each other and tossed a coin to select a winner. We would eventually add on several more “classic” games including trivia, roulette and jackpot and made them provably fair when it was applicable.

The Tech

The front end was written in Angular, the back-end was written in Node.  I had several years of experience in Node development prior to this and front end experience with Angular and React, so I was actually the most experienced developer on the project, though the team was extremely capable. We used RethinkDB as our back-end store, and Redis for caching. SocketIO was used for all real-time client updates. I wasn’t responsible for deployment, but we used Google Cloud (Like Amazon Web Services) with Kubernetes and Docker containers. We were basically ready to scale in an instant. We could deploy new pods, update existing pods, and scale out without ever causing a disruption to the site, it was really cool.

The Business

Whats interesting about sites like these is that you never actually deal with “real money”. It still works like an online casino, you offer games and you take a rake, but players don’t deposit money, they deposit virtual items: “skins”. They get compensated with an in game currency based on the current market value of the skin, and they use this to play games.

We kept track of the current value of each type of skin so we could value our entire inventory.  We used several third-party skin trading sites for price data.  We also kept track of the amount of in game currency on all users accounts. Our profit was the difference of these numbers.

Every so often our sales guy would take skins from inventory, equivalent to our current profit, and sell them in bulk to a reseller at a discount of 50-70%. The reseller would buy the skins with Bitcoin. Our sales guy would then pay the team in Bitcoin. That was the whole business.

So what were the numbers? Once our site was peaking around 1000 concurrent players every day (about 2.5 months after launch), i’d estimate we would rake from the games about 10-20k per day.  There were other daily costs, giveaways, bonuses, trivia prizes, advertising, exploits etc. There were the cost of running servers, and the discount and depreciation we took when selling skins. So after all of that take home profit was maybe half of that, 5-10k per day which is still amazing for a team of 5.  We didn’t have much time to enjoy that though, as we shut down the site about a week after reaching those numbers.

The Challenges

Launch

We launched in my opinion, before we were ready (according to startup philosophy, that’s what you should do).  We had about a month+ of development then a 1 week long alpha, and a few days of beta. The marketing team was ready with a couple hundred people to use the site when we launched. Right off the bat we had issues with the trading system and with the games, but releasing like that helped us identify problems right away. We realized we needed a support system, so that got introduced early on. We cruised along with a hundred or so people for a week or two, and got a big bump up to 500 at some point, then hit our next stumbling block.

Scaling Games

Our flagship game,  Bust, was really laggy once the active players started hitting around 500.  Its a game that requires very responsive controls since it is a time critical game. We were doing a lot of database calls and Redis calls to make sure game state was synchronized across servers.  Eventually had to refactor the game to run all in memory through Redis. This was the first refactor which allowed us to handle 500+ players, but we would need to revisit performance again eventually.

Hack Attack

Shortly after release, we were hacked. We never really figured out how they did it, but we had some ideas. The hacker was able to login as an admin and send in game money to players.  The direct way to login as an admin was to login through Steam, which was a 2 factor authentication system on a commercial site. Probably not possible. The only other way we could think of is if the hacker somehow got a hold of the session data of an admin.  We didn’t know which admin, because we weren’t logging any of that data.  So we undid all the unauthorized transactions, added logging to those routes and added https to encrypt data to and from the server.  We didn’t have that problem again.

Market Manipulations

This was actually an on-going problem we never fully resolved. Our site allowed users to trade in their skins, and we would compensate them with an in game currency equal to what those skins were valued at on the market. The problem is the market can be low volume and extremely volatile. Prices could fluctuate or be delayed, and a single player could manipulate the prices of the market.  The markets that we used to price our items were third party sites. Errors in their price API’s could also be issues for us.  If there was a mis-priced  item, we could be sure users would find it and exploit it.  Over the course of our site being up we probably lost several thousands of dollars just to people making smart trades ( or actively manipulating prices), depositing over valued items to us while withdrawing undervalued items. We added a withdraw “tax” among other rules to help prevent losses, but we never in my mind found a bullet proof solution.

Scaling Games Again

It was a big achievement when we hit 1000 players. That was kind of a big goal from the beginning. We were still a new site at about maybe 2 months in, but now we were sort of medium sized.  The problem was that Bust could not handle the load, the back-end and front end were becoming unresponsive with the amount of players playing. It was critical the game be lag free.  We had to visit the front end code and do some optimizations, while the back-end needed a different approach. I eventually redid the entire architecture of the game which allowed it to scale well past 1000 players and also simplify the game logic greatly. For the most part this new architecture gave us a pattern for easily adding new games scalably and we never had issues with performance after that.

Hacked Again

About 3 months after launch Valve made a big announcement denouncing gambling sites. Shortly after that we experienced another hack attack. It was sort of a one-two punch to our morale, and again, we never really figured how it happened.  The hackers struck twice in succession, making off with some 20k worth of skins.  Its hard to know exactly how much, but it was significant.

The hackers somehow got direct access to our Steam bots. Each bot is basically a regular Steam account which were used to store skins for users, but they were automated, so we could use them to send and receive trade requests programatically. Getting access to the bots would be the same as getting access to someones Steam account, nearly impossible. Somehow they had logins, passwords and 2-factor authentication capability for several bots.  There were some strange things the hackers did leading up to the attack, so its uncertain if this was an exploit, a security lapse on our part, or an inside job.   I don’t really know enough details, so I won’t speculate further.

Shut Down

The last hack attack caused our site to disable all our bots while we changed our custom bot handling code to use steamtrad.es, a third party skin inventory management site. This was all moot as a week later Valve issued a cease and desist to sites like ours.  The reaction most of us had was sort of a silent acceptance, not too much was said. I think we all knew it would happen eventually, but it was still pretty sad.

We kept our withdraw system open for a week so users could cash out, but we disabled all games and eventually shut the site down completely.

The End

We all worked really hard on this project, and ironically from the beginning I knew it was a gamble and didn’t expect the ride to last very long.  I’m amazed we were able to become as successful as we did and in such a short time, and I’m really proud of all our work on the site. I appreciate the opportunity to have been brought on to a project like this, and the guys I worked with were really great and dedicated.  I’m also thankful to all the players who trusted us with their time and money and hope they had a good time while it lasted.

With anything like this where you really put your blood and sweat into, it is disappointing to have it end so abruptly, but hopefully there will be more projects to work on.

I might eventually do a more technical article on the system architecture of the site, so stay tuned.

Advertisements