Levyx and Neueda Partner to Bring Python Into the Modern, Big Data Era
IRVINE, Calif., August 05, 2019 -- Levyx Inc., a leading provider of ultra-low latency data processing software that brings high performance to demanding enterprise applications, announced today its partnership with Neueda, the leading-edge data management specialist, to deliver key enterprise enhancements to the Python community. By adding these critical features through the Helium–Python interface, the parties have increased Python’s utility and applicability in use cases that are expanding in number and volume.
Today’s Python implementations are hampered by the difficulty of handling increasingly large data sets inside current in-memory objects, thereby limiting the language’s utility in quickly modernizing and scaling financial trading environments. To address these and other limitations, Levyx and Neueda have worked together to connect Levyx’s Helium data store into Python. The result is a new dictionary object that can extend to many Terabytes on a single server. By re-using the native Python dictionary interface, financial quants can manipulate, store and access datasets (for example, tick data and order data) that are an order of magnitude larger than what is currently feasible, using a familiar interface. This is important since “Pythonistas” typically process their data on an end-user workstation and the Helium-Python interface eliminates the tedious process most quants go through of breaking up their financial data sets into manageable sizes.
Brendan Monaghan, CEO of Neueda, said: “Levyx’s technology is playing an important role in transforming conventional data processing in financial markets. We are excited to bring the power of the Helium data store to Python, allowing our customers across the globe to handle larger data volumes than ever before within their applications. We are delighted to be able to add this to our growing suite of products for the electronic trading community through https://blu-corner.io.”
Reza Sadri, CEO of Levyx, said: “We are pleased to work with Neueda on upgrading Python with an important new feature set, which generally enhances its enterprise-worthiness, for Fintech users and other verticals as well. Neueda’s customer platform is an excellent way to reach Python users who can leverage these advances.”
In recent testing, Levyx and Neueda compared the performance and other attributes of a native Python dictionary vs. a dictionary that uses Helium. While Helium maintained performance comparable to the in-memory native Python dictionaries, it provided the following additional powerful support attributes:
Larger dictionaries (6 times larger on a typical laptop; 30 times larger on a commodity server);
Persistence across different programs and multiple invocations of the same program; and
Efficient data sharing between different Python programs and processes that leverage Helium’s ultra-efficient locking and concurrency mechanisms.
In addition to accommodating larger data sets, the Helium–Python interface is useful for concurrency. In particular, different Python programs typically need to signal to each other that they are accessing a certain key or key range, and this new implementation, backed by Helium, allows multiple Python programs, even when running on multiple servers, to access the same persisted dictionary with performance similar to that of native in-memory Python dictionaries.
From a practical standpoint, this means that teams of quants can use the solution to allow a single copy of a large multi-Terabyte core data set to be used via a server interface by many researchers simultaneously. This eliminates needs for duplicate copies. The solution also provides for archiving results of quant or backtesting model research efficiently. According to one manager of several dozen quant researchers, the solution effectively eliminates his need to negotiate with IT for more server resources. Researchers can deal with large datasets right on their workstations.
Levyx software is hardware-independent and can be used by almost any system, including the high-end servers found in large enterprise datacenters, all the way down to workstations and even much smaller devices like cell phones. In datacenter use cases, Levyx software technologies serve as enhancements to or replacements for many of the world’s most popular Big Data / open-source packages, including Python, RocksDB, Apache Spark, LevelDB, Java Hibernate, and Memcached. Levyx’s Helium engine is one of the world's fastest key value stores and is optimized for flash memory, storage-class memory, and multi-core systems.
About Levyx Inc.
Based in Orange County, California, Levyx designs high-performance software solutions to accelerate analytic and computational functions of demanding enterprise applications on any storage media or platform. Optimized for the latest hardware innovations, Levyx disrupts the economics of running big data applications and brings the benefits of high-speed data processing to all enterprises. The company’s core value-add is to enable customers to process their massive workloads in real-time at fractions of the cost of competitive solutions. More information on Levyx’s stand-alone or embeddable software products is available at www.levyx.com.
About Neueda
Neueda is a trusted partner to international trading firms and fintech vendors. Based in Belfast, Dublin and Malaga, Neueda’s team of trading technology specialists help firms to develop and maintain applications for the global markets in the evolving exchange and regulatory landscape.
Source: Levyx Inc. and Neueda
