Skip to main content

Market and Reference Data Management: A Maturing Art

Managing Market Data Resources: A Multi-Dimensional Challenge

Optimizing Market Data Management Resources

A common concern of Market Data Management Teams (or those accountable for a Market Data portfolio) is the number and disposition of resources to employ for effectively managing Market Data spend. Unfortunately, the answer is never a simple linear relationship of bodies to contracts that most expect to hear. Rather it is a multi-dimensional problem that requires detailed understanding of the composition and complexity of services; quantity of users and contracts; and skills available to the organization in managing vendor and internal business relationships.

The number and variety of services consumed by financial firms is enormous. Over 3000 services from over 500 vendors globally are available to serve the range of data requirements from real-time to reference data users across the entire firm. The variety ranges from simple, pre-packaged services and content (e.g. Market Data terminal) to expansive data feeds and sophisticated processing platforms that demands participation by other parts of the business. Each serve different consumer interests and have dramatically different impacts on Data Management activities.

Size of an organization in terms of number of users and variety of businesses are also important to understand. These factors directly contribute to the complexity of the technology needed to service users. An example of this is a common quote terminal. While as an individual product or package it is easily understood and manageable, en masse it develops complex distribution network issues as an organization grows both numerically and geographically.

Lastly, the skills and their distribution within an organization have a significant bearing on resources to employ within the Market Data Management function. The more dispersed throughout the organization, the more challenging relationship management becomes. It is common to observe unwanted behaviour as services and management of those services become more complex. Specifically, the dismissive approach taken by a business unit or technology group having unique knowledge of a sophisticated or high profile service. These situations, are the proverbial “canary in the coal mine” of a failing market data management function that is both costly and wasteful.

By understanding the complexity and number of Agreements within a firm, one can position the required skill level (or key practice) to apply for: developing strong relationships (business and vendor), managing User Demand, and optimizing the Operations & Technology role in delivering market data solutions.

The Market Data Management Maturity Curve

To understand how organizations should resource the Market Data Management function, Sia Partners has developed a maturity curve that correlates the capabilities required with the nature of an organization’s Service Provider Agreements. These capabilities are bucketed into key practices that a firm must adopt as they increase their relationship management dependencies.

Market Data Management Maturity Curve

The Market Data Management function consists of multiple activities (or key practices) demanding a variety of skill-sets. Determining how many, and what mix, of resources to employ in performing these activities first requires an understanding of the nature of Agreements within an organization.

Quantity and Complexity of Agreements

Quantity alone does not speak to the complexity and skills required to procure, manage, and sustain an optimized Market Data services environment. This is the common trap most firms fall into as they assume vendor products, and the associated contract management process, is homogeneous. With over 3000 services globally, many having distinct terms and conditions, the breadth of knowledge and experience required to facilitate market data service optimization is vast.

  • Commoditized products – Market Data Terminals, News, and End-of-day Pricing services are commonplace in the industry. These products are readily understood and standardized in many respects including how they are packaged in providing solutions. They can be readily managed with basic “order management” activities. However, quantity of users and location can add to the complexity of those services as unique skills are required to facilitate a large distribution network and demand management tactics to track utilization.
  • Niche Products – Risk Management, Valuation, low-latency datafeeds, and Research services are also commonplace, however they have incredible variety and levels of sophistication, even within their own category. They often require specialized skills to procure, they regularly require other groups to engage (e.g. Technology), and they have few direct competitors to compare against to determine if value is being received.

Depending on the Commoditized vs. Niche mix, and number of Agreements within a firm, will affect the slope of the curve. This is commonly observed as a distinction between sell-side and buy-side firms. Niche products are more often observed in Capital Markets based businesses accompanied by bigger budgets and highly specialized skills – steep curve. Commoditized products tend to be in the domain of Wealth Management and Pension Funds with more Commoditized services that aren’t viewed as requiring similar capabilities – flatter curve. Regardless, Sia Partners emphasizes having transparency and a clear understanding of the number and the complexity of those Agreements (collectively and individually) before any assessment of optimization or skills gaps can be made.

Capability or Capacity to manage Agreements

Managing internal as well as external relationships that result from market data contracts require distinct skill and time. The key practices that follow the curve upwards and to the right speak to the increasing need to either acquire the resources needed to facilitate the practice, or coordinate internal groups who bear the skills required.

Much like quantity of contracts was described earlier, time (or time to process renewals and manage agreements) is not homogeneous for all contracts. Even common “Administrative tasks” can vary in terms of time to process with the complexity of an agreement, or when a change to a service term occurs at renewal. For example, an above-normal price increase at renewal calls upon a different set of skills to facilitate the best decision – product knowledge, business relationship management, and negotiation skills to best position or even substitute the service. The Market Data Management team requires the time and skills to be able to properly execute.

However, the suggestion is not to concentrate all the necessary skills (and time to execute) in one group, especially considering the expertise and knowledge often already exists within other parts of the firm (e.g. complex, technology-heavy market data solutions). The suggestion is that the market data management function expand to include bodies skilled at managing internal relationships to draw out product and business intelligence necessary to fulfill their mandate (manage market data consumption, compliance, and costs). This is where a skills gap typically emerges, as the existing team has neither the skills nor the time to leverage the expertise available – the capacity to execute does not exist. This is most obvious in an organization when the Business side-steps the “normal” procurement process, or Technology inadvertently creates a liability by making access “better”. Left unchallenged, it leads to:

  • Service Creep – excess, often duplicated services
  • Knowledge Dispersion – decentralized intelligence gathering and management

Resourcing a Market Data Management group using the Maturity Curve

Ironically, it is only upon engaging in Spend Management or value-add programs (a practice engaged by a mature market data function) that a firm can understand where their skills gap exists. Detailed analysis of how many, and how complicated relationships are help determine where to layer in key practices to be employed.

The key practice layers employed as a firm moves across the maturity curve

Key practice layers

The challenge for every market data function is not for simply recognizing where a skills gap exists (most firms typically know this already). The challenge is for quantifying the size of the gap in terms of cost, and creating the impetus to develop a systemic approach to manage market data on a regular basis. The common behaviour displayed by firms is a degree of complacency until a significant market shock (e.g. 2001 Tech Bubble, 2008 credit crisis) or dramatic strategic change (e.g. merger/ acquisition) occurs. This of course mobilizes the resources who come in at the head-end with a Specialized or Spend Management program often assuming that the previous key practices are sound and intact. Unfortunately these are short-term strategies with random savings objectives. Rarely is the intent to develop a systemic approach for Market Data Management.

At Sia Partners, the emphasis is on using Spend Management and Specialized programs as the vehicle to identify gaps in providing two advantages: a) short term savings or “quick wins” to fund the analysis, but more importantly b) developing and augmenting the skills with strategies that helps a firm position their Market Data Management function within the context of their third party commitments. The attention is for cost optimization, effective processes and practices, and greater utility from the relationships built upon the services employed.

Spcialization: Extensive, organized Analytics for Integrated Decision Support

Transcending the Curve

Assuming only one Agreement, Market Data Management requirements are minimal. The product and business knowledge required to make effective decisions are optimized as the key stakeholders are directly engaged in the process. Inventory tracking requirements are non-existent given the quantity, and likely better served by the vendor through their invoicing records. And the only support a User requires, for the most part, is that bills get paid. The core elements of Administrative Support are sufficient in this environment.

Of course firms have more than one Agreement, and Administrative support alone will become overwhelmed (or provide minimal value) as numbers of Agreements and Users increase. Allocating costs, managing budgets, and maintaining compliance with Agreements becomes more important to the firm as growth needs to be managed. This requires a tracking mechanism, or inventory management system to provide the necessary transparency into services, users, and costs throughout the organization. An Excel spreadsheet or proprietary database typically serves the fundamental requirement of managing a small number of Agreements (50 to 75). And while some will use these to manage many as 100 (depending on the complexity of those Agreements or number of users) there is a point at which more sophisticated tools are required to achieve operational advantages – Databases/ Applications that automate vendor report ingestion; Invoice reconciliation; and detailed user and service information for advanced reporting capabilities.

For no apparent reason, many firms stall out at this level of the maturity curve. This is presumed to be a resource constraint, where the required product knowledge and business relationship skills required for the next Agreement tends to be sought from other parts of the organization. The skills and knowledge required are recognized and addressed, however it is distributed throughout the organization with no coordinated structure to ensure the individual parts are meeting with enterprise goals.

The simplest example is a Market Data Distribution system (Bloomberg M-Pipe or RMDS) where Technology takes on a dominant role for the implementation and operation of the system. Technology clearly has the Product Knowledge and capacity to execute, however their focus is on the fixed or “boxed” component of the relationship (where market data is the variable component with theoretically unlimited risk). With no offense to the many Technology professionals that are critical to the Market Data industry, charging them to manage this aspect of the business alone is equivalent to expecting a Plumber to manage your water bill. There is a logical connection, but an unrealistic expectation that User Demand, Administrative effectiveness, and Supply (Vendor) Management can be optimized.

This is the level, above which, most skills gaps are observed. And unfortunately, the gap can go undetected for an extensive period of time as the simple reality is: Invoices get paid; Users have budget; and Technology is more than capable of delivering the next challenge. Firms will often continue down this path until either a major market shock occurs (leading to the dreaded call to “find savings”), or some new technology is introduced that motivates a firm to revisit their current environment. In other words, absent the ongoing analysis with skilled product and business relationship staff, firms aren’t aware of the potential efficiencies and cost savings available to them.

Only through the routine analysis of every contract at renewal, the regular engagement of the business to understand their requirements, and the benefit of time to gain an organizational perspective on the impact of an Agreement; can the market data management function fully satisfy the organizations expectation - to optimize market data within the firm.

Note that while the curve depicts Product Knowledge as the next key practice or skill-set to advance, developing out Business Relationships takes precedence where the environment consists primarily of commoditized products (many users but non-complex agreements/services). Regardless, both are closely related and required to achieve the next level in the maturity curve – Spend Management. For clarity, spend management is not simply a cost-savings exercise. It is better thought of as a series of advanced programs that allow an organization to develop key metrics, and asses the value of their market data vendor relationships.

Whether it is the number of services a firm manages, or the growing complexity of the services it is moving toward, building upon the advanced product and business knowledge acquired within a Market Data management function is necessary to achieve a mature state. The reward for a firm moving to its full potential of a highly skilled Market Data Management function – efficient Demand Management, Supply (or Vendor) Management, Ops&Technology Support, and Administration Management.

Skills/Resources Impact of Increasing Third Party Dependence

Management and Technology Consulting for Today's Business Environment

Sia Partners is an experienced, client-focused provider of consulting and transformation services dedicated solely to the financial industry.
Our experienced professionals help organizations map out and execute data management solutions that create enterprise value.

Contact us to learn more

Sia Partners integrates this data in its client database to send you marketing communications (invitations to events, newsletters and new commercial offers).
This data will be kept for 3 years before being deleted and you can withdraw your consent to the processing of your data at any time.
To learn more about the management of your personal data and to exercise your rights, please consult our Data Protection Policy.


Your data are used by Sia Partners to process your contact request. Please note that you have rights regarding your personal data. For more information, we invite you to read our data protection policy