Driving Innovation, Agility, and Greater Business
Returns Through Automation on AWS
to technology and enabled its traders to make quicker and
more successful business decisions through an automated
reporting and analytics platform developed by Stelligent on AWS.
How Hedge Funds Turn to Digitalization to Gain a Competitive Edge
Speed matters for hedge funds. The quicker a business decision is made from insightful data, the higher chance a
hedge
fund will stay ahead of the competition as it drives favorable outcomes for its investors and managers. The right
combination of data, knowledge, security, speed, and strategy can establish a hedge fund as an industry leader and
innovator. In order to give traders the access and information they need to make logical decisions for the billions
of
dollars they control, hedge funds around the world have an intolerance for technical debt. Instead, the companies
embrace the on-demand compute power of cloud providers such as Amazon Web Services (AWS) and advanced big data
analytics
solutions running on AWS.
Hoping to gain massive amounts of scalability and agility while also driving consistent and secure delivery
processes,
one hedge fund brought in Stelligent, an AWS Premier Consulting
Partner and AWS DevOps and Financial Services Competency
Partner, to develop
an automated reporting and analytic
platform so their teams could launch and deploy a over 100-node
Splunk analytics platform with the click of a button.
Turning to Stelligent to Automate an over 100-Node Splunk Cluster on AWS
Splunk Enterprise monitors and analyzes machine data from any source to deliver Operational Intelligence to optimize a
company’s IT, security, and business performance. By running Splunk Enterprise on AWS, the hedge fund knew it could take
advantage of the on-demand scalability, availability, and world-class security standards of AWS. The team also wanted to
proactively build its solution on AWS in an optimized fashion from day one and sought to deploy its Splunk application
on AWS in a fully automated manner to test quickly, make changes efficiently, and build and validate new components
rapidly.
Looking for experts who embrace a codify-everything approach to development, the hedge fund engaged Stelligent to build
a Continuous Integration (CI)/Continuous Delivery (CD) pipeline for the company’s full technology stack,
including AWS resources and resource configuration and its over 100-node Splunk cluster. The team wanted the ability to script
everything so that a production-ready cluster was automatically built with the click of a button.
Stelligent began by walking the team through its particular methodology for architecting the hedge fund’s
infrastructure, what that would look like, and how it would affect the company’s development team’s method of building
and deploying their environments. Stelligent then embedded with the company’s development team to help drive and
architect the portions of the CD pipelines in the Splunk automation. Once the Splunk piece was in place and the company
could centrally consume all infrastructure and application data and query the data, then Stelligent helped the team
implement a more standard methodology for CI/CD for other applications to take advantage.
The hedge fund solution employs a multi-Availability Zone (AZ) and multi-AWS Region high-availability architecture and
creates various levels of accounts based on data classification standards. AWS services used include:
The team also uses Buildbot, an open source CI tool, and HashiCorp Terraform provisions all of the AWS services using
infrastructure-as-code.
Security is unequivocally the hedge fund’s number one priority, and the company continuously sought ways to put security
mechanisms around everything its teams create without hampering development teams. Stelligent used and expanded on AWS
security best practices as it helped the hedge fund take an “everything-as-code” approach to its infrastructure. In
doing this, the hedge fund could make changes through scripts and see what was built along with how it was built as it
adds all changes and deployments to its code repository. Through this approach, the hedge fund can efficiently run
audits, and security teams can make sure everything is as intended.
Given Splunk’s ability to consume security and operational-related data metrics within AWS, such as VPC flow logs,
CloudTrail API calls, and AWS Config data, by deploying Splunk in an infrastructure-as-code manner the company is able
to correlate security data with its application data seamlessly. This ability provides the hedge fund with complete
control and visibility over all of its data sources and enables its teams to make queries against the wealth of data
collected.
Scaling Rapidly to Process Data Quickly
Working with Stelligent to take advantage of automation on AWS, the hedge fund can test and iterate quickly and develop
new innovative modules that affect how they report and approach analytics and consume trader data. And all of this is
done in a highly available, easily replicable fashion. From a security perspective, running on AWS improves the hedge
fund’s security posture given the firm’s immense visibility into each component of its architecture and the team’s
ability to audit everything. By using Splunk on AWS, the security teams can cross-reference different activities of the
application logs and the infrastructure logs. If there were a malicious or accidental activity within an area of the
application or infrastructure, the security team could quickly identify it through its ability to cross-reference data.
Over time, the hedge fund went from having a single application running on AWS, its Splunk cluster, to having almost 100
different applications running on AWS. By embracing an automate-everything mentality, the hedge fund can now seamlessly
codify and launch new infrastructure and applications on AWS to drive innovations and more profound insights to its
traders.