Learn how one global hedge fund took an automate-everything approach
to technology and enabled its traders to make quicker and
more successful business decisions through an automated
reporting and analytics platform developed by Stelligent on AWS.
How Hedge Funds Turn to Digitalization to Gain a Competitive Edge
Speed matters for hedge funds. The quicker a business decision is made from insightful data, the higher chance a hedge fund will stay ahead of the competition as it drives favorable outcomes for its investors and managers. The right combination of data, knowledge, security, speed, and strategy can establish a hedge fund as an industry leader and innovator. In order to give traders the access and information they need to make logical decisions for the billions of dollars they control, hedge funds around the world have an intolerance for technical debt. Instead, the companies embrace the on-demand compute power of cloud providers such as Amazon Web Services (AWS) and advanced big data analytics solutions running on AWS.
Hoping to gain massive amounts of scalability and agility while also driving consistent and secure delivery processes, one hedge fund brought in Stelligent, an AWS Premier Consulting Partner and AWS DevOps and Financial Services Competency Partner, to develop an automated reporting and analytic platform so their teams could launch and deploy a over 100-node Splunk analytics platform with the click of a button.
Turning to Stelligent to Automate an over 100-Node Splunk Cluster on AWS
Splunk Enterprise monitors and analyzes machine data from any source to deliver Operational Intelligence to optimize a company’s IT, security, and business performance. By running Splunk Enterprise on AWS, the hedge fund knew it could take advantage of the on-demand scalability, availability, and world-class security standards of AWS. The team also wanted to proactively build its solution on AWS in an optimized fashion from day one and sought to deploy its Splunk application on AWS in a fully automated manner to test quickly, make changes efficiently, and build and validate new components rapidly.
Looking for experts who embrace a codify-everything approach to development, the hedge fund engaged Stelligent to build a Continuous Integration (CI)/Continuous Delivery (CD) pipeline for the company’s full technology stack, including AWS resources and resource configuration and its over 100-node Splunk cluster. The team wanted the ability to script everything so that a production-ready cluster was automatically built with the click of a button.
Stelligent began by walking the team through its particular methodology for architecting the hedge fund’s infrastructure, what that would look like, and how it would affect the company’s development team’s method of building and deploying their environments. Stelligent then embedded with the company’s development team to help drive and architect the portions of the CD pipelines in the Splunk automation. Once the Splunk piece was in place and the company could centrally consume all infrastructure and application data and query the data, then Stelligent helped the team implement a more standard methodology for CI/CD for other applications to take advantage.
The hedge fund solution employs a multi-Availability Zone (AZ) and multi-AWS Region high-availability architecture and creates various levels of accounts based on data classification standards. AWS services used include:
The team also uses Buildbot, an open source CI tool, and HashiCorp Terraform provisions all of the AWS services using infrastructure-as-code.
Security is unequivocally the hedge fund’s number one priority, and the company continuously sought ways to put security mechanisms around everything its teams create without hampering development teams. Stelligent used and expanded on AWS security best practices as it helped the hedge fund take an “everything-as-code” approach to its infrastructure. In doing this, the hedge fund could make changes through scripts and see what was built along with how it was built as it adds all changes and deployments to its code repository. Through this approach, the hedge fund can efficiently run audits, and security teams can make sure everything is as intended.
Given Splunk’s ability to consume security and operational-related data metrics within AWS, such as VPC flow logs, CloudTrail API calls, and AWS Config data, by deploying Splunk in an infrastructure-as-code manner the company is able to correlate security data with its application data seamlessly. This ability provides the hedge fund with complete control and visibility over all of its data sources and enables its teams to make queries against the wealth of data collected.
Scaling Rapidly to Process Data Quickly
Working with Stelligent to take advantage of automation on AWS, the hedge fund can test and iterate quickly and develop new innovative modules that affect how they report and approach analytics and consume trader data. And all of this is done in a highly available, easily replicable fashion. From a security perspective, running on AWS improves the hedge fund’s security posture given the firm’s immense visibility into each component of its architecture and the team’s ability to audit everything. By using Splunk on AWS, the security teams can cross-reference different activities of the application logs and the infrastructure logs. If there were a malicious or accidental activity within an area of the application or infrastructure, the security team could quickly identify it through its ability to cross-reference data.
Over time, the hedge fund went from having a single application running on AWS, its Splunk cluster, to having almost 100 different applications running on AWS. By embracing an automate-everything mentality, the hedge fund can now seamlessly codify and launch new infrastructure and applications on AWS to drive innovations and more profound insights to its traders.