Custom Affiliate Marketing System
Project details
This particular custom affiliate marketing system provides an effective set of tools to navigate and operate within a large variety of incoming information in real time. The main purpose of the product is to monetize providing a viable platform for ads designed to drive mobile app installs.
Our client had an existing CPA product and they were preparing to expand their business, so they needed to convert it to a white label marketing platform so that the system was capable of handling the Planned traffic growth. Therefore, our client approached us with a request to build a custom affiliate marketing system aimed at mobile conversions.
Our mission is to extend and refine the toolkit for data processing — making it scalable, adding new parameters and metrics, making the design easy to follow.
Challenge: Migrate the system to the cloud
One of the biggest challenges we tackled during application development was migrating to a new system built on Google Cloud Platform . We need to ensure a smooth transition and scalability. It involves transferring several terabytes of data from a previous platform. Not only do we need to design an efficient inclusive architecture from the start, but we also need to make it scalable and highly functional.
Challenge: Create a data analytics feature
That challenge is combined with adding many new features into the mix. To make targeting and data analysis more accurate and effective, we have added several new parameter types and diversified the metrics. This has improved and stabilized the performance of the traffic control and targeting components.
As we continuously add new features, we need to ensure continuous deployment without loss of operations on the customer side. The main purpose of these additions is to expand the scope and make the overall picture more detailed.
Challenge: Develop data tracking for high loads
Another difficult task is handling the tracking component. To improve performance and safe scalability, we rewrote the tracking component so that it can handle large numbers of requests.
The challenge: Make it usable for everyone
We also needed to give the system a user-friendly interface that wasn’t too complicated to master. With a little help of UX principles, we managed to make it as intuitive and comfortable as possible.
Our solution
Our first step was to migrate the system to Google Cloud Platform. While working to scale, we rewrote the data processing and tracking components. This increases the stability of the system and makes it more reliable.
In its current form, the system combines high efficiency and intuitive design. From a monolithic product, we are moving towards services that offer more flexibility. We have a powerful admin panel with multiple levels of user permissions.
The system itself is targeted at conversions, rather than the traditional targeting of impressions and clicks. Therefore, the monetization scheme is built to maximize conversions. Enterprise features include sophisticated and precise targeting as well as a variety of tools that make data and campaign management convenient for end users.
Currently, the system can handle over 100 million events per day with approximately 5000 queries per second at peak load.
Continuous integration is organized through a blue-green deployment scheme , allowing for the seamless addition of new functionality with no downtime. This is achieved by using two sets of servers for the product. One set is active while the other is kept on standby as a release candidate. When the new version of the product is ready — the servers are swapping.




technology stack
To provide the best possible way to develop applications, we tried several software options before settling on a complete suite. We wanted to create an application whose maintenance would be completely independent of additional software.
The following tools have been used to ensure smooth, uninterrupted operations within the Android platform:
- monitor:
- Symfony 4 and PHP 7.1
- Google Cloud for auto-scaling and cost reduction features
- Google Stack Driver for monitoring;
- Redis Cluster for quick access to operational data.
- Data analysis and processing:
- Apache Beam on Google Cloud Dataflow for scalable streaming data processing;
- Java — used for data processing;
- Initially, we used MySQL for interactive analysis of huge datasets. Then we turned to BigQuery because of its integration.
- Google Cloud PubSub — real-time messaging service.
- Terraform, Packer, Ansible – toolkits for infrastructure automation
- PagerDuty – integration with this service provides a 24/7 support and alert system to monitor system health;
Our team includes:
- 1 Project management
- 3 Backend programmers
- 1 Front end developer
- 1 QA engineer
- 1 DevOps Engineer

Summary
This project was a huge technical achievement for our team. The entire project’s architecture has been revised, upgraded, and the project has been migrated to Google Cloud to simplify the scaling process and reduce costs. Considering the complexity of the project and the requirement for easy scalability, we chose Google Cloud services.
The backend is also optimized to handle an average of 2,000 queries/second (peak load to date is around 5,000 queries/second). For this project, we implemented an innovative solution for ad targeting with the ability to choose different IP database (IPDB) implementations for each white label (Maxmind and Digital Element). digital). This decision also positively affects cost efficiency.
Thanks to our partnership, the client now has a platform where its customers can subscribe to new CPA targeting systems. Analytics now include more than 50 parameters with 20 metrics, describing each impression, click, and conversion. All of this is presented in a simple, intuitive interface.