Grindr Settles into a Scalable Platform to Expand its Range of Services
What separates a platform from just a bunch of software? For one, a platform can allow a business to easily expand into new services and markets, without a corresponding pain in expanding IT.
This might be the lesson gleaned from the how mobile social networking service Grindr is managing its IT. A wildly successful mobile-only geosocial networking application built for helping gay and bisexual men hook up with like-minded others nearby, Grindr is planning to parlay its services beyond the hook-up, to cover a range of lifestyle needs. Think of a Grindr for tasty restaurants nearby, or a Grindr for finding a good show.
With its new platform in place, such expansions are all that much easier thanks to the platform’s extensibility.
“We have a tremendous opportunity. We’re talking about a large, large ecosystem of users that we’ve built over the years,” said Lukas Sliwka, chief technology officer of Grindr. “The vision is, in the next couple of years, to essentially turn our platform into a place where a gay man can go and find out about not only who is around him but also about what’s going on around him.”
Sliwka started there about two years ago when the company only had four software engineers (it’s now up to 40). When he first signed on, Sliwka wasn’t initially worried about expanding into new markets. Rather, he had a much more urgent mission: stabilizing the company’s platform, and hitting some SLAs (service level agreements).
As a social networking service, Grindr is huge. The service has about two million active daily users, and they use the service a lot. Each user spends about an hour a day, on average, occupied on the site, more than the average daily user time on Facebook. Presently, the peak usage for the company is around 3:30 P.M. pacific time on Sunday, where over a million users can hit up the service. This can translate to between 10,000 to 20,000 APIs requests, and about 1,5000 to 2,000 chat messages, crossing the wire per second.
First Task: Establish an SLA
When Sliwka came onboard, the platform was based primarily on Ruby-on-Rails, and ran a “bunch of different custom technologies that the team built from scratch,” Sliwka recalled.
As a result, all this custom work amounted to “an infrastructure where that was really expensive to manage and maintain and also was fairly brittle,” Sliwka said. “Because it was custom infrastructure, it didn’t have the benefits of regular updates.”
For instance, Grindr, being a location-assisted service, needed strong geospatial algorithms, which the company created itself, even though Google offered perfectly suitable algorithms. Another feature, chat, was originally cobbled together with Jabber.
“You don’t want to be wasting the cycle of these people to manage things that are already solved for” — Lukas Sliwka.
Sliwka’s first task was to find those home-built components that could be replaced with some easier-to-manage out-of-the-box technologies.
Jabber was replaced with a more manageable MongooseIM stack, which was supported by Erlang Solutions. The company also brought in ObjectRocket, which basically packages MongoDB as a service so it can be easily deployed.
Within a year, after refactoring clients and gearing towards the “crush-rate” or highest possible volume of traffic, the team was able to maintain a consistent SLA, with a minimal number of outages.
Next Step: Architect a Scalable Platform
So, in the middle of 2014, Sliwka took the next logical step, namely to map out a more scalable architecture. He gathered an architectural team and had them hide away on a retreat to build a blueprint of what would be the company’s “new stack,” to be implemented over the following year.
Gone would the Ruby-on-Rails, replaced by a set of technologies the emphasized non-blocking I/O, including Akka a highly scalable Scala-based framework.
For this framework, the team considered both Node.js and a Go framework, though the company feared they would not be able to find enough developers to cover the technology at least not at the scale at the level Grindr would need.
“I did not feel like we had the ecosystem of developers that had the necessary experience that we could hire from,” Sliwka said. “And I can tell you that, even with Java, it’s much easier to hire Java people who have worked at scale because Java has been around for so long as an enterprise. Even with Java folks, it’s not an easy thing to find engineers that are familiar with solving issues at this scale.”
For configuration management, Grindr went with Apache ZooKeeper. Amazon Web Services provides the infrastructure, so applications are packaged and deployed by way of Amazon Elastic Beanstalk.
The company uses the open source RabbitMQ for back-end messaging, but rather than trying to maintain it in-house, the company procured the services of CloudAMQP. For the caching layer, Redis Labs provided a highly-scalable Redis cluster. Sauce Labs and Appium are used to automate most all of the functional testing.
A company called Treasure Data handles the data capture, ingestion and encryption. MongoDB, which can be a bear to scale, was shed for many duties in favor of Elasticsearch. To help keep latency to a minimum, Grindr uses the CloudFlare content delivery network (CDN) for static material such as text and photos.
The idea behind all this outsourcing is to “minimize the dependency on our own internal DevOps team,” Sliwka said.
“One of the things that I’m very big proponent of, especially for startups or midsize companies is that, as you grow you want to make sure that your engineering department focuses on the things that contribute the most towards that competitive advantage. You don’t want to be wasting the cycle of these people to manage things that are already solved for,” Sliwka said.
By June 2015, the new stack was up and running. Now, the next step is to redefine the APIs to extend the functionalities to new user-facing features. Many of the services were, or are being, rearchitected into microservices, so they can be used in multiple use cases.
With the infrastructure in place, Sliwka is now able to concentrate on getting the right data for the additional services. Grindr is now hiring a data science team, and “scaling out our data processing capability to be able to start applying some of the deep learning into the data that we’ve been collecting over the last year,” Sliwka said.
“It’s pretty exciting because the approach that we’ve taken has been to really focus on building a platform, focus on providing microservices and reusable components, “Sliwka said. “So then as we’re building all these different pieces, then the dating and meet-up engine is just one application of that platform for a company.”
