TrackStreet needed a data platform with the processing power to quickly ingest data at internet-scale as well as store and analyze that data into information so TrackStreet’s customers could quickly understand pricing behavior in their supply chains and act to enforce policies.
Background
Price management is never a set and forget program; success or failure is determined every day. The explosion of online resellers multiplies the challenges of brand management and pricing policy enforcement. Without enforcement, policies are at risk across supply chains, where decisions and actions of each distributor, dealer, or reseller create the potential for non-compliance, undermining the decisions of pricing strategists and putting revenue projections in jeopardy.
Detecting violations at internet-scale and intervening in time to correct and stop contagion are challenging disciplines.
Trackstreet delivers pricing policy enforcement and brand protection as Software as a Service by offering technology that constantly monitors online marketplaces to record prices and identify unauthorized resellers. By comparing current prices with historic prices in visually stimulating ways, the technology accelerates understanding and empowers fast and effective responses to protect brand value and grow revenues.
TrackStreet: Bringing Order to Pricing Strategy
The market-leading pricing policy enforcement and brand protection platform from Nevada-based TrackStreet enables every brand with online sales capabilities to protect and grow their multi-channel revenue. Delivered as a Software as a Service platform, TrackStreet leverages artificial intelligence and automation technologies to relentlessly monitor the pricing of millions of sellers on over 100,000 websites and eCommerce marketplaces.
Rising to the Data Challenge at TrackStreet
TrackStreet’s success at putting the technology needed to execute effective pricing strategies into the hands of brand and pricing professionals created its own challenges.
“Originally our platform was built using the open source MySQL database hosted on AWS,” explained Rene Manqueros, Principal Software Engineer at TrackStreet. “This worked well until we hit a data volume threshold and all database operations slowed to levels where we became concerned with our ability to service our customers. We could have increased the instance size on AWS but the uptick in cost made that option economically infeasible. Moving to Elasticsearch for searching and reporting provided relief from escalating costs of managing our data, but its impact was temporary.”
Rene and the team realized that TrackStreet’s success at proving the value of pricing policy enforcement and brand protection to a growing customer base meant the time was right for a data warehouse modernization project.
Data Warehouse Modernization: Assessing the Contenders
Rene and his colleagues assessed and tested CockroachDB, TimeScaleDB, and MongoDB but these technologies could not fulfill TrackStreet’s requirements. The warehouse assessment at TrackStreet was reduced to two contenders: a leading cloud data warehouse and Yellowbrick.
“The leading cloud data warehouse appeared a promising technology,” Rene recounted, but “pricing was always hidden, explained away as ‘pay as you go.’ So, we ran our proof-of-concept. The cost was absurd.”
Working with Yellowbrick was different. TrackStreet discovered the team to be transparent and open.
“One of the Yellowbrick team members asked me for some details on my setup. The next day he had a solution for me. Seeing the real technical level of the people assigned to work with us and their commitment to go that extra mile made this a very different vendor relationship.”
Testing Yellowbrick
“We decided to test Yellowbrick,” said Rene. “I loaded some historical data on a test instance assigned to us. The process was fairly simple and fast. In less than two hours Yellowbrick completed a load job that had taken me several days with the other DBMS we considered. Converting and running the report queries on Yellowbrick was a breeze, as the queries originally were written in MySQL. In less than three hours I had changed keywords to make them psql-compatible. On the other DBMS we assessed, this conversion had taken several days.
“Next, we ran the big test case,” he continued. “On Elasticsearch this would take around two minutes to process. On Yellowbrick it completed in five seconds. I increased the load to 20 times the original and Yellowbrick completed its work in about one minute. This was the scaling behavior we needed; it confirmed that Yellowbrick would work for us as our company kept growing.
“Then, we tested our data patching use cases, which update 1-2 billion rows. Again, the Yellowbrick team stepped up, offering invaluable advice. The update completed in under fifteen minutes. We would never have attempted this as a single process on MySQL; the row locks could destroy the instance. With Elasticsearch we had to chunk the updates and run them in small batches across a period of one month. So, a huge win for Yellowbrick.
“Our final criteria was to evaluate how easy it would be for our developers to work with Yellowbrick,” explained Rene. “Most of them know MySQL and have some exposure to Elasticsearch and MongoDB, so Yellowbrick’s Postgres-compatibility made it an easy choice. Converting our Elasticsearch code to Yellowbrick was really simple. Additionally, this gave us an opportunity to increase the number of rows processed from a thousand in Elasticsearch to ten thousand in Yellowbrick. Because Yellowbrick radically reduced the computing time required, we have downscaled our Amazon Elastic Container Service cluster from 40 to 5 nodes.”
Yellowbrick Performs TrackStreet’s Heavy Lifting for Price Management
“As soon as we had Yellowbrick in production as the warehouse handling all our data, all TrackStreet’s other teams were ready to migrate their reports,” said Rene. “We started cautiously, just to see how the new warehouse would perform, but soon realized that Yellowbrick was taking it like a champ. Now, whenever some data processing is slow, everyone’s first suggestion is: move it to Yellowbrick.”
As to the future, he noted, “TrackStreet is considering migrating all analytical work to Yellowbrick.”
As the discipline of price management develops and evolves, TrackStreet receives a constant stream of requests from clients for custom reports. Rene concluded, “Our team sees Yellowbrick as the best destination for these requests.”
“This was the scaling behavior we needed; it confirmed that Yellowbrick would work for us as our company kept growing.”