Why Infrastructure Observability Still Lags, Says the Engineer Who's Tackling It Head-On

When Nikhita Kataria talks about infrastructure observability shes not speaking from a nonskininthegame viewpoint as Nassim Taleb likes to call it As a Site Reliability Engineer at LinkedIn shes spent her career buried in logs metrics and system behaviour shaping observability systems not just to monitor but to improve how core infrastructure functionsKataria has built out a series of observability tools and processes that are now part of infrastructure initiatives at LinkedIn One of her significant achievements includes the integration of a system that autogenerates metrics and alerts for control plane servicescomponents that manage rather than serve userfacing data These services power everything from deployments to configuration changes making them critical to platform reliabilityAnother major area where her work has made an impact is in infrastructure debugging Kataria developed a Grafana dashboard that traces events across different control plane services helping engineers pinpoint what action was taken on which data center host and by whom This crossservice traceability she says drastically cut down mean time to debug from days to just hoursHer work isnt limited to diagnostics Shes also made observability proactive At LinkedIn she led the effort to track OS upgrades across all data centers The dashboard she built provides realtime insight into upgrade progress and failure ratessomething that was previously tracked manually or retroactively This tool is now used daily and serves as a reliable view into infrastructure hygieneOn the operations front Kataria worked on alert tuning for LinkedIns deployment stack improving the signaltonoise ratio and helping reduce alert fatigue among engineers She also created operational review dashboards that bring together key business and system metricsSpeaking of metrics the metrics for the successful implementations are clear Her OS upgrade tracking dashboard is in daily use showing percentage completion across hosts and helping teams prioritize fix efforts Her work on creating a Grafana dashboard to trace all events for a data center host across different control plane services is being used by key offline systems powering LinkedIns website and by defining SLAs SLOs and SLIs for critical control plane services shes helped teams focus on what matters to users and the SWEs had a clear goal on what are they supposed to improveBut these activities didnt come without their own set of challenges One project required unifying log schemas across more than 10 different teams Each team emitted events in a different format making it nearly impossible to run meaningful queries or correlate activity She not only initiated the schema standardization effort but also managed stakeholder alignment ensuring the unified structure was implemented across the boardIn another initiativedesigning disaster recovery metricsshe faced the issue of knowing what would be required to track in the first place Researching the key indicators needed to monitor both compute and storage health for which she built a framework for service owners to ensure readiness in critical failover scenarios A paper based on this work is currently pending publicationFrom her standpoint observability is not a postlaunch featureits a design principle If a software engineer says we can think about metrics later they are signing up for operational toil later she says According to Kataria thinking about observability from day zero can prevent months of reactive troubleshooting down the lineHer perspective also takes into account whats next Artificial intelligence she says is going to reshape monitoringbut not without limitations People are building AIpowered features but AI still requires a tight feedback loop to learn the right observability patterns she explains Even similar tech stacks across companies like Facebook and LinkedIn generate different monitoring needs depending on how the software is used in practice Moreover AI would need a deep understanding of operating systems to set meaningful thresholds or surface relevant anomaliesLooking forward she sees observability becoming more embedded not as a standalone concern but as a core pillar for a healthy infrastructure architecture With her work already influencing how missioncritical services at LinkedIn are built and monitored shes clearly helping that infrastructure take shapeone metric at a time

Why Infrastructure Observability Still Lags, Says the Engineer Who's Tackling It Head-On

Other Articles

Mid-Day Fast

Home / Buzz / Article / Why Infrastructure Observability Still Lags, Says the Engineer Who's Tackling It Head-On

Why Infrastructure Observability Still Lags, Says the Engineer Who's Tackling It Head-On

Other Articles

Mid-Day Fast