26 Oct 2023
Uncategorized Permalink

Google BigQuery Demystified: A Short and Sweet Overview for Data Professionals

2023-10-26

Alright folks, buckle up because we’re about to dive deep into the world of BigQuery, Google’s behemoth in the data warehouse domain. So, what’s the deal with BigQuery? This is not your granddad’s database; it’s a fully managed data warehouse that’s all about giving you the luxury of focusing solely on churning out SQL queries to dig up business insights while it takes care of the dirty work – think deployment, scalability, and security. It’s like having a butler for your data, fancy, right?
Now, for the love of clarity, let’s not mix up data lakes with data warehouses. Imagine a data lake as this vast, wild ocean of raw, untamed data. It’s just sitting there without purpose, probably having an existential crisis. On the flip side, a data warehouse, and yes, I’m talking about BigQuery here, is like a well-organized library, with all data placed neatly on the right shelves, ready for some serious querying action.

BigQuery, my friends, is not a data hoarder. It’s in the game for the thrills of analysis and helping you make those million-dollar business decisions. It’s like the Flash of data analysis; it can run through terabytes of data in seconds and make petabytes of data spill their secrets in minutes. Imagine having 350 petabytes of data at your disposal, like one of Google Cloud’s customers. That’s like trying to drink the ocean with a straw, but hey, BigQuery makes it happen.

Now, let’s talk architecture. BigQuery sits at the end of your data pipeline like a king on its throne, overseeing the incoming data and ready to jump into the analysis and model-building arena. It’s a two-in-one powerhouse: a lightning-fast SQL query engine and a fully managed storage layer. It’s like having your cake and eating it too. And the best part? It separates the brawns (compute resources) from the brains (storage), allowing you to store and analyze your data on your own terms.

Key features? Oh, BigQuery is loaded. It’s got machine learning, geospatial analysis, and business intelligence all built in. It’s serverless, meaning you don’t have to worry about the nitty-gritty of resource provisioning or server management. Just roll up your sleeves and get down to business with your SQL queries.

Pricing? As flexible as a yoga instructor. You only pay for what you use, whether it’s the bytes of data your query processes or the permanent table storage. And if you like to know precisely what you’re getting into every month, there’s a fixed billing option too.
Security? Check. Your data is locked up tighter than Fort Knox, encrypted at rest without you having to lift a finger. And machine learning? It’s like having Einstein in your tool kit, ready to crank out ML models directly in SQL.

Now, let’s get our hands dirty. Dive into the BigQuery user interface, and let’s run some queries on a whopping 10 billion rows of data. Yes, you heard me right. This is different from your average data playground. As you run that query, watch BigQuery flex its muscles, scanning and processing data at the speed of light.

So, what have we got here? Datasets, tables, and columns are all neatly arranged and available at your disposal. And slots, those virtual CPUs that BigQuery uses to run your SQL queries like a well-oiled machine. Don’t worry about provisioning; BigQuery’s got your back, dynamically allocating resources based on your usage patterns. You use it. You pay for it. As simple as that.
In conclusion, if data is the new oil, BigQuery is the super-refined gasoline that’s ready to fuel your business’s rocket to success. So, go ahead, take it for a spin, and watch your data transform into business gold. Welcome to the big leagues, my friends. Welcome to BigQuery.

15 Dec 2022
Personal Development Permalink
A mentor or coach gives someone a hand to get back up on their feet after an intense workout
Photo By Alora Griffiths

5 Reasons Why I Want to be Mentored by Paul Randal

2022-12-15

[Original Post 20221215]

  1. Development: I need help in professional development and community exposure. The value of taking action on the advice, insight, and wisdom from someone who has been down that road and cares enough to point out my blind spots and places to level up is priceless. I want to increase my influence within my teams and with co-workers. I want to be a high-value asset to my organization.
  2. Emails: It can take me 45 mins to write a two-sentence email. 
  3. Documentation: I overthink and need help quickly asking and answering the question, “who is my audience”.  
  4. 10X My Effort: I want to 10x my efforts in 2023 with accountability. In January 2019, I attended my first SQLSaturday in Nashville and found out that anyone could submit to speak at a session. I submitted my session a month later, and from June 2019 – Feb 2020, I taught 17 sessions at 13 SQLSaturdays around the country. And in Nov 2022 gave my first presentation (virtual) at PASS Data Community Summit. I am forever thankful for the opportunity those people gave me. I am a go-getter who usually gets things done using brute force and tenacity. I need advice and strategy from someone who is blunt, honest, and does not sugar-coat things. In short, please tell me what I need to know and point me in the right direction. 
  5. Give back. This mentorship program will increase my network by getting to know the other mentees and help give me the resources and opportunities to give back to a community that has given me so much opportunity and guidance within my personal and professional life. 

[Why I wrote this Post]

Paul S. Randal of SQLSkills has opened his mentorship program for 2023 and I’d like to be a part of it. The most important requirement to be considered was for me to write a blog post explaining why I would like to be mentored and then post the link in the comment section of the blog post. The deadline was Thursday, 15th December, at 23:59:59 PST, and I’m happy to say that I posted the link with a few hours to spare.

Photo by Alora Griffiths on Unsplash

[Update 20221228]

It’s official, I’m on the list for the Paul Randal Mentoring class of 2023. If you read the post, he tries to be a little sneaky and shady. Paul first writes that he can only take 15 of the 33 who put their names in the hat. In my first reading of the post, it seemed that he listed everyone who “applied” and then listed the 15 people who got accepted. So, to see if I got picked, before reading the entire post, I simply searched for my name. If my name came up twice then I was in the program. Well, my name only showed up once and I thought, “at least I put my name in the hat.” Well, Paul flipped it on us and took everyone. I’m grateful for the opportunity and I’m looking forward to what I learn and implement. I’ll make sure to document the process at least via blog posts.