Cloud computing is a fast-growing and dynamic field that requires constant learning and updating. As an international marketer, I am always interested in discovering new tools and techniques that can help me improve my cloud operations and observability. That’s why I was delighted to attend the workshop on cloud observability at Google Amsterdam on October 11, 2023. It was a one-day event that presented the latest trends and best practices for collecting data, troubleshooting applications, and monitoring infrastructure at scale in Google Cloud. In this post, I will share some of the highlights and learnings from the workshop.
The Workshop Agenda
The Cloud observability workshop started at 9:30 am and lasted until 5:00 pm. It was an in-person only event, which gave me the opportunity to meet and network with other cloud enthusiasts from different countries and backgrounds. Furthermore, the workshop was led by @Afrina and @kylebenson from Google USA. They were deeply knowledgeable and experienced in cloud operations and observability. Besides, they also had a lot of humor and charisma, which made the workshop more engaging and fun.
The workshop agenda consisted of several topics, each with a presentation, a demo, and a Q&A session.
Here are some of the topics that were covered:
- Best practices and trends while building reliable observability pipelines. This topic introduced the concept of observability and its benefits for cloud applications. It also explained the difference between metrics, logs, and traces, and how to use them effectively to monitor performance, availability, and reliability.
- Troubleshooting complex applications using cloud logging and monitoring. This topic showed how to use Google Cloud’s logging and monitoring services to troubleshoot complex applications.
- Built-in tools and techniques to monitor Kubernetes clusters. This topic focused on how to monitor Kubernetes clusters running on Google Kubernetes Engine (GKE). It explained how to use GKE’s built-in tools like Stackdriver Kubernetes Engine Monitoring, etc. It also showed how to use third-party tools like Prometheus and Grafana to collect and visualize metrics from Kubernetes pods and nodes.
- Best practices for optimizing observability costs. This topic discussed how to optimize observability costs while maintaining high-quality data and insights. It shared some tips and tricks on how to reduce data ingestion, storage, and processing costs by using techniques like sampling, filtering, retention policies, exclusion filters, and log sinks. It also explained how to use billing reports and cost management tools to track and control observability spending.
- Operationalizing SRE principles using Google Cloud operations suite. This topic introduced the concept of site reliability engineering (SRE) and its principles and practices for building reliable systems. It showed how to use Google Cloud operations suite (formerly Stackdriver) to implement SRE concepts like service level objectives (SLOs), service level indicators (SLIs), error budgets, incident management, postmortems, and continuous improvement.
My Key Takeaways
The Cloud observability workshop was very informative and practical. I learned a lot of new things about cloud observability and how to use Google Cloud’s tools and services to achieve it. Here are some of my key takeaways from the workshop:
- Cloud observability is not only about collecting data, but also about making sense of it and using it to improve system reliability and user experience.
- Cloud observability requires a holistic approach that combines metrics, logs, traces, events, alerts, dashboards, reports, etc.
- Cloud observability is not a one-time project, but a continuous process that involves planning, implementing, testing, analyzing, optimizing, etc.
- Cloud observability is not a solo effort, but a collaborative one that involves developers, operators, managers, customers, etc.
- Cloud observability is not a static goal, but a dynamic one that evolves with changing requirements, technologies, environments, etc.
Conclusion
The Cloud observability workshop at Google Amsterdam was a valuable and enjoyable experience for me. I learned a lot of new things and met a lot of new people and I gained a deeper understanding of the latest tools and techniques for collecting data, troubleshooting applications, and monitoring infrastructure at scale in Google Cloud. I also explored the latest trends and how to use them to incorporate innovative ideas into my operations.
If you are curious about it, you can attend the next workshop that is going to be held in Munich on October 16, 2023 (Be sure to reserve your place). Special thanks to the Google Cloud Community and to the community manager, Lauren_vdv, for organizing this workshop and for providing such a great learning opportunity. I hope to attend more workshops like this in the future.
Some memories from the Cloud observability workshop:
Frequently Asked Questions
Observability is the ability to understand the internal state of a system by examining its outputs.
The three pillars of observability are metrics, logs, and traces.
Observability can help you to:
• Identify and troubleshoot problems more quickly
▪ Improve the performance and reliability of your systems
• Reduce downtime and costs
▪ Gain insights into how your systems are being used
Some common observability tools include Prometheus, Grafana, and Jaeger.
To build a reliable observability pipeline, collect the right data, store it in a scalable way, and visualize it in a meaningful way.
To troubleshoot complex applications using observability, collect the relevant data, analyze it to find the root cause of the problem, and fix the problem.
To monitor Kubernetes clusters, use tools like Prometheus, Grafana, and kubectl.
To optimize observability costs, only collect the data that you need and use the right tools and techniques for collecting and storing data.
To operationalize SRE principles using observability, collect the relevant data, monitor your SLOs (Service Level Objectives), and investigate and resolve SLO violations.
Some of the latest trends in observability include the use of AI and ML to automate observability tasks and the adoption of SRE principles.