How is Database Formed?

Borrowing from an ‘old meme’ a bit.  Somebody recently told me I should “write something about ‘how to do databases’.”  As amusingly odd as that phrasing was, I figured they were right.

Obviously. GAWD.

What is it?

I like to start at beginning.  As Julie Andrews said, it’s a very good place to start.  What is a database?  That’s a pretty good question.  Here’s the prerequisite question: What is data?  Well, as I’ve said before, data is everything.  But that’s a bit of a cop-out, isn’t it?  That’s my career’s bias showing through.

Data is digital information.  Anything that can be quantified, specified, categorized, searched, sorted, produced, consumed, read, written, measured, and stored digitally in some fashion.  Data is the digital currency of the 21st century.  Data is the very reason that most technology exists — to house and transport data among producers and consumers of information.  It’s the evolutionary culmination of the stone tablet, the papyrus scroll, the bound book, the printing press, the newspaper, the library, the vinyl record, the magnetic tape, the compact disc, the pocket organizer, and the telephone.

So then, what is a database?  Simply put, it’s a collection of data.  The simplest analogy, depending on your age, is either a phone book or your cell phone’s contacts list (which is really just a phone book, in digital form).  Of course, with the latter, it’s not so much an analogy as an example — you phone’s contact list IS a database.

Fun side-note, the phone book also makes a decent discussion prop for some DBA topics like index fragmentation.

Expanding on that example.  You can search and sort your contacts by several data points: first name, last name, phone #, email, notes.  Different database systems have various names for these: fields, columns, properties, criteria, values.  The point is, it’s all data.  Or if you want to get pedantic, each one is a datum, and together they are data.

Pedantic, me?  Never.

This is what a database, or DB for short, is all about: storing data in an organized fashion so that it can be sorted, searched, sliced and diced.  Building on that, a database management system is a set of technology tools, processes and programs, that are used to gather, store, manipulate, copy, move, read, maintain, back up, link together, and operate one or many databases.  This DBMS can come in many flavors.  I happen to specialize in one called SQL Server, a Microsoft product/platform of the ‘relational‘ flavor — so if you’re following along with the abbreviation game, that’s an RDBMS.

If you’re hungry for more acronyms, the Wiki article on ‘databases‘ has a decent breakdown of the types and history behind them.

But Why?

Non-geeks might not understand this, so Google “all your base” if you’re confused.

The more data you have, the more you can do with it.  Why do you think Facebook, Google, Microsoft, and Amazon are such powerful technological forces?  They purposefully, systematically gather as much data as they can from every possible source, and they have become very good at organizing and managing that data to maximize its value.  Amazon product recommendations are a prime (see what I did there?) example — they are generally appropriate and effective because they have “learned” from your past purchases, i.e. your historical data.  This “learning” – Machine Learning, aka Data Science – is the hot new marketing buzzword of recent years, but it all still comes back to data at the core.

This is not a “bad thing” or a “scary thing” as the old media and tin-foil-hat-wearers would have you believe.  Yes, it can be a little disconcerting, and yes, people and companies can abuse data in malicious ways.  But the vast majority of our digital data stewards actually want to do good.  They want to connect you with more people that you may know and become friends with; they want you to watch movies that you’ll really enjoy; they want you to easily navigate to your destination without being stuck in traffic; they even want to help stop global warming!

As a general business rule, we crave data because it helps us make decisions.  Every time a customer buys a product, we want to measure “the 5 W’s”: who what when where and how (ok, that’s not a ‘W’, but there’s a reason for it).  Notice I didn’t list “why” there — only the customer knows why, and that information, that data, is stored inside their brain.  And we can’t (yet) access that data.  So it’s a guessing game now — we feed the other 5 data-points into our DBMS and eventually, given some time and analysis, we can guess the Why.  And pretty accurately, at that.  Then, we can make a decision to “Market more aggressively to Customer Type X”, or “Have a flash-sale on Product Y”, or “Move on this hot emerging market demographic.”

So what does that make you?

Well, I’m a Database Administrator – a DBA.  Which means I “administrate databases”.

‘Administrate’, less common form of ‘administer’: manage and be responsible for the running of.

Thanks, dictionary.  So in a nutshell, a DBA manages data.  Deceptively simple sounding, no?  I mean, what can data possibly do; it’s not alive, right?  Actually, if you hang around a DBA for any length of time, you’ll commonly hear the phrase “Where does that data live?” or “That set of data lives over here.”  So clearly we anthropomorphize our data.  Most tech professionals do that to whatever technology they work closely with — it’s human nature.  Software “behaves badly”, machines “throw a fit”, etc.

But anyway, why do databases need to be managed?  What can happen to them?

Developers.  Developers happen.  =D

I joke, as you know, dear reader; I love developers.  Users ‘happen’, too — often more catastrophically.  So it’s fair to say that “people happen”.  But besides that, here are some common reasons that databases, and data, need to be managed.

• Data can be “wrong”.

Data can either be human-generated or machine-generated.  Fingers on a keyboard, or sensors on a circuit board.  You wouldn’t think the latter could possibly ever be “wrong”, but both kinds are subject to error.  It’s just that the level of “wrongness” is subjective and depends on who’s asking and what’s expected of the system as a whole.

• Data gets lost.

Humans interact with and manipulate data, and humans make mistakes.  Why do you think the Undo button became such a staple of so many computer applications?

• Data gets corrupted.

Storage media (magnetic disks, silicon chips, etc.) are not perfect — they have a documented level of fault tolerance and failure rate — so we need to ensure that our data is preserved (by moving it to another area that’s not ‘faulty’, usually) past those failures.  Why?  Because our data is essentially “more valuable” than the hardware on which it’s stored.

• Data needs to be organized.

This is slightly more subjective than the above; how and why we organize data is highly dependent on the overall intent of the systems that will interact with it.  But fundamentally, if there’s not some form of organization, the data is effectively garbage.  If you ripped out every individual page in the phonebook and scattered them all on the floor, it’s no longer an effective tool to find someone’s phone number; it’s just a mess of papers.

• Data needs to be useful.

If we can’t do something with the data, what’s the point of having it?  The temperature at the North Pole on January 1st 1989 is, by itself, inconsequential.  But a history of temperatures at the same and similar locations, over a long period of time, gives us some great value — we can see trends, look for anomalies, and even predict the future of what those temperatures might be.

• Databases need to be available.

Similarly, if we can’t access the data, what good is it?  Databases are a technology, and like most technologies, they occasionally break.  Again, most of that comes back to humans, because humans are the ones writing the code that creates the software that houses the data and runs the database, or that interacts with it.  But of course we still have power failures, network losses, disk failures, and even solar flares.  (Ask your favorite superstitious engineer; they’ll have at least one good story about a system outage that could only be blamed on solar flares or gremlins or the full moon.)

• Databases need to be maintained.

Every DBMS has some kind of assumed ongoing maintenance requirements to keep it “running smoothly”.  Just like your car needs an oil change every 3 to 8 thousand miles, your databases need periodic attention to retain all of those important qualities discussed above.

And the latest big topic, underscored by the GDPR:

• Data needs to be governed.

This is a big topic for another conversation, but the gist of it is, data is generally “owned” by someone, and deciding who owns what, where it’s allowed to live, and how it’s allowed to be used, constitutes an entire sub-industry of rules, regulations, policies, tools, security practices, and consequences, much of which we’re only just beginning to shape and understand.

TL;DR: What do you actually do?

Umm… stuff? And thangs?

I currently work at a “small enterprise”, a business that has been around for some decades (as opposed to a Silicon Valley start-up who counts their anniversaries in months, like an infatuated teenager), managing their database systems.  Some of that is financial/accounting, some is customer info, some is internal/operational, and all of it is important to at least one person in the business in their daily decision-making efforts.

Thus, I help ensure that the data is always ready, when it’s needed, in whatever form & shape it’s needed in.  I model, massage, correct, enhance, and move it around.  I help developers write faster queries (that’s a fancy word for “questions” that we ask of our data); I aide analysts with understanding and gleaning more value from the data; I maintain the underlying systems that house the databases and ensure that they perform well and get upgraded when necessary; and I work with business drivers (VP’s, CxO’s) to build reporting solutions that leverage the data to enable better, smarter decisions, and ultimately (hopefully!) increase profit.  (This last part is actually crossing into the BI – Business Intelligence – job role, which tends to happen to most small-shop DBAs, because they’re usually in the best position to make that transition.)

If some of that sounds like a blurb from a résumé, it kinda is.  This job has existed since the 80’s.  But it’s always evolving, like the tech industry in general; so just because we’ve been around a while doesn’t mean we’re all old crusty bearded dudes.  (Although we do have some prolific beards among us!)

So there you have it.  Now you can tell your friends and family what a DBA does.  Or at least, hopefully, I’ve helped my own friends & family understand a bit about what I do.

Off-Topic: A Short Story

Moved to new blog, go check it out!

Mistakes We Made in Not Hiring Women

This is partly inspired by the recent #StackOverflowPodcast episode in which Jon Skeet and the some of the Stack Overflow women talk about the Feminist movement and what it’s means to them.

It’s so exciting!

So I’m not going to even “go there” in terms of the movement in general, where I stand, or anything really deep, because it’s a huge iceberg of a topic and I can’t do anywhere near justice to it.  Even the discussion above was pretty basic & high-level, but they give lots of links in the show-notes to truly deep-dive into, if you feel like it.

One burning question, essentially, boils down to this:

Why is there still a striking disproportion of females in the Data Professionals career space (and, more broadly, the IT/Dev space in general)?

And like so many questions I ask here, I won’t actually have an answer for you (that would be too easy, wouldn’t it?).  Instead, I’ll simply reflect on my own (very limited) experience in working with women in tech, and hopefully get your thoughts in the comments as well!

Early Colleagues

In a very small software shop, there were actually no women for a time.  But shortly before I was hired, they brought on a husband & wife team – two developers who were excellent at what they did, and had, surprisingly, broken the stereotype of “spouses can never work together”.  She was so easy to work with, and I think my experience here helped me at least start to understand some of the nuances around “women in tech” and what it all meant.  Basically, she wanted to be treated just like “one of the guys” — which, I understand now, is, in itself, an anti-feminist phrase, but back then I wouldn’t have known — and it reflects the culture of the time, which is that this was a team of mostly male developers and we were still “finding our way” on the long trail of equality in the workplace.

So what this meant, in practical terms, was a couple things:

• No bias for job-assignments of things like documentation, task management, or communication-centric tasks.  While yes, she was actually quite good at these, and would later become the de-facto PM and Scrum-master for us, it was understood (and probably stated) that this was not “because she was female”, this was because she was the best at it.  But again, is that specifically because she’s a woman?  I don’t think so.
• Addressing the group as “you guys” was perfectly acceptable.
• Pay was equal – at least between the equivalent roles & seniority levels (e.g. her & her spouse).  You don’t typically share or discuss salaries among peers, but we knew, and our bookkeeper ensured, that this was true.  Because if it wasn’t, someone would’ve had some words about it.

Also, there were a few positive aspects of the culture that helped make it more equality-apparent, which I think were just byproducts of the quality of people hired.  We didn’t do “dirty jokes” or have sexist (even unintentionally) discussions, nor did we engage in gossip or any kind of “just the guys” activities.  We really just did the work and kept it professional, and any time we were outside the office together, it was almost always shop-talk.  I think that’s the nature of a startup — you really don’t have time for anything else, any of the “fluff” or crud that spawns from idle hands & minds.

But it wasn’t all roses & sunshine.

A New Female Developer Candidate

After that dev moved on, we knew we had to replace her.  And the company workload was pivoting a bit, so our candidate criteria weren’t the same as those of her position.  But putting it that way makes it sound like we were either specifically looking for someone different, or that we had moved somebody else into her position and now had a completely different role to fill.  Neither is the case, really; with a startup that’s organically growing and shifting, you don’t get the luxury of well-defined roles.  You basically need what the business and the team needs at the time, and that becomes your reality, until it’s not, and your team pivots again to fill the new mold, learning & growing along the way.

So anyway, we were hiring for a somewhat nebulous developer position.  And one of the candidates we saw was female.  We did not end up hiring her — unfortunately, in my opinion.  That’s not to say the candidate we did hire was bad; he was great too, that’s just not relevant here.  After her interview, the discussions we had were interesting.  And I think it would have been greatly beneficial if the previous dev (the woman I talked about above & who had left recently) could have been present to offer her insight into the hiring process; but she, understandably, was not available & already busy with her new stuff.

This new candidate had a good deal of “embedded systems programming” background, which was interesting because it was not at all what our software was about, but in hindsight, probably could have proved valuable in making our SDLC processes leaner & more efficient.  She also had great general tech skills and was a quick learner.  But ultimately the reasons not to hire came down to the dissimilarity of background vs our product, AND her personality as we perceived it — in a word, she was “nervous” and “not confident”.

This is a big failure, in terms of equality/feminism.

And as I said, this is all purely hindsight.  None of us realized at the time what we actually meant.  But that’s no excuse, just history.  So let’s unpack that a bit.  Or, in other words…

Ya think?!?

DUH!!

Of course she was nervous!  She was A) in an interview, and B) surrounded by men.  It’s not like we said anything or acted in a way that was actually misogynistic; we’d like to think we’d learned how to be open & equality-centric enough that anybody would feel welcome and able to talk about their experience and why they’re a good fit for the job & the company.  We didn’t even have much of a “culture” to speak of — it’s not like we were a big enough team to even have cliques or established norms, we just kinda discussed our work, did the work, collaborated on the work, and went home at the end of the day to our families/friends.  However, in the same breath, we DID have a “culture”, in the sense that we were a small tight-knit team (while in the office) with a set of personalities that, so far, worked very well together; and on a small team, personality-compatibility is important.

Anyway, here’s the crux.  We didn’t even recognize that what we were saying was, underneath, an anti-equality statement:

She should have been more self-confident than the average male candidate, in an interview, in order to meet our expectations.

Now obviously, if you ask the hiring manager (aka owner/CEO/president/founder) of the company and the HR person, they’ll agree (rightfully so) that she was not hired due to the gap in technical experience & the fact that her skills did not fit with what we needed.  And this is true, as I said before; we were doing web-based software in ASP.NET, with a SQL back-end, and none of those were at the top of her skill-set.  So I’m not self-flagellating for us passing on her as a candidate.  (Again, the person we did hire worked out just fine.)

I’m acknowledging, and apologizing for, the fact that we elevated her (completely understandable) personality/disposition to an artificially high importance in our discussion about the candidate.

That, I think, is what an equality-minded leader would try to make sure we avoid next time.  If she had had very similar experience and skills to the next candidate, we should have certainly hired her.  And if I were to retroactively predict the past-future (breaking all kinds of grammatical rules in the process), had she been hired, she’d have “come out of her shell” and gotten along swimmingly with the team & the work.

But again, this is all ancient history by now; it’s not like we can go back and change our decisions.  We just need to be conscious of them and learn from them.

“But I’m trying…” / “Don’t make me say the line again.”

The Passing of the Torch

It’s always hard to say goodbye to a colleague, especially someone who’s so central and ingrained in the company lore and holds so much of the “tribal knowledge”.  Hell, I was that guy just a couple years ago.

Can you just leave your whole brain right there on the desk? Thanks.

So now I’ve seen a couple such old-hats move on from my current team, and seeing both sides of the proverbial torch-passing is interesting.  There’s definitely some very common, very important things that we should always do.

Documentation, documentation, and more documentation.

Indeed.  Also, finishing critical tasks, handing off in-flight projects, re-assigning tickets, talking to managers, prepping teammates for the work overflow, and cleaning out that huge buildup of clutter that you’ve collected over the years.  Virtual or physical… often both!

Unsurprisingly, where we all seem to differ widely is the human aspects.  Breaking the news, saying goodbyes, doing those last-minute get-togethers and send-offs.  What do those last few weeks and days look like?  For some, it’s just business-as-usual up to the last minute — they’re literally so busy they have little other choice.  That’s how it was with the helpdesk manager we parted with last year.  I used some of the time to put together documentation and thank-you letters, which I hope ended up being helpful.  Database diagrams were printed and taped.  Wikis were written.

But the main thing is to make sure you exchange contact info and stay in touch.  It gives the team a sense of comfort, knowing they can reach back out when those random questions that nobody’s thought about for several months resurface.

KITSA!

I’ve learned a lot from those folks that took the time to pass on their knowledge and made the effort to keep in contact.  And I appreciate them for that!  Today I’ll thank one of my exiting managers; she knows who she is.  She taught me a lot about our internal application stacks, integration and interop, company culture, tribal knowledge, and not standing for anybody’s BS, including my own.  Good luck with consulting, stay in touch, and kick some butt!

That’s all for this week.  I promise I’ll work on that “database collation problems” post soon…  :o)

TSQL Tuesday #96: Good Influences

This month’s invitation is brought to you by Ewald Cress (blog, twitter), who I already like based on his tagline —

finds joy in minutiae..

Yes, my friend, don’t we all.

I can’t spend the rest of my life coming into this stinking apartment every 10 minutes to pore of the excruciating minutiae of every single daily event!

The topic at hand is fairly non-technical, but still important: folks who have made a positive contribution to your career or professional development.  So it’s time for a shout-out!  About a year ago, I wrote about my first major career move.  There were several great influences in my first job, from the developers that taught me how to code, to the DBA who taught me how to keep cool & calm in the face of outages, to the boss who taught me the importance of time management and breadth of knowledge.

Post-mortem

Since I am way too late in posting this, and I don’t feel like waxing poetic, I’ll just say a general “thank you” to all those who’ve helped me along in my career so far, with special acknowledgement to my former boss, my current boss, the SQL family, and my own family.  Happy belated Thanksgiving and have a safe & pleasant holiday season!  I’ll have a real post again quite soon, diving back into the tech stuff.

hope they don’t mind me borrowing their image… =D

Who Am I?

I started out as a data analyst, first playing in MS Access, and quickly moving to MS SQL Server. I also became a developer in the ASP.NET stack, way back in the 2.0 days (2006-07). Yet I found myself drawn to the data more than the code, for some reason. Perhaps it was the beautiful simplicity of the relational model. Or the myriad of variations that something as simple as an name field could contain & how challenging it could be to massage that “dirty data” into something elegant and consistent for the code to easily manage and present to the user. Or it could have been the thrill of analyzing a workload that may have taken several minutes to run, and then, by applying the right combination of indexes, relations, and set-logic, to bring that workload down to milliseconds.

This job was great. First “real” job out of college; a teeny-tiny software shop, 4 devs & the boss, working with K-12 school districts, making their lives just a bit easier when dealing with the vast swath of standardized testing and government-imposed student performance metrics that ultimately decided how they taught & how they got funded. And we grew, slowly but surely – 5 devs, then 6, then 8. We got to wear so many hats! Developer, Data Analyst, Helpdesk, Tech Support; even Printer, Delivery Crew, Trainer. It was truly a well-rounded experience.

As the years went by, it felt like family. And sure, every family has the occasional episode of dysfunction. But we were in it together, building up the business with passion, commitment, and respect. And those of us that were there since the start, we felt like we knew it all. Because we did; in that little microcosm, we truly were the experts. And each of us had a forte – I was the “data guy”, he was the “self-taught developer”, she was the “tech-evangelist / UX dev”, and he was the “dev architect”. Our best hire came soon after – the “young enthusiastic developer”, a true professional. Then there was our den-mother, bringer of snacks, paycheck-writer and book-keeper extraordinaire. And as always, the father-figure – the big kahuna, the boss man, who was more than willing to get his hands dirty and help with anything we asked him to.

That was a wonderful experience. In the near-decade I spent with that company and those colleagues, many of whom I consider my friends, I learned so much about technology, programming, databases, and business. And we welcomed more and more people into the family – the graphics & print-media design expert, the bright & cheery project manager, the ambitious salesperson, the kind & caring tier 1 support staff, the new data analyst, the veteran DB-dev, the expert architect, the UI/UX dev, the student dev, and even the Russian programmer! (They’re amazing, BTW.) Each & every one of them added value, and taught me something new.

I do not regret a single day of it. It was awesome.

But, as the baby bird has to eventually leave the nest, I needed to venture out, to get outside my comfort zone and see what else was out there, to join a new & different team and environment, to see if I was truly as good as I thought I was. Well, I think we all know the answer to that! (Hint: it’s “nope”.)

Actually, it’s been great – I drank from the proverbial fire-hose, learned a lot in a short time about a completely new line of business & a familiar but larger & much more complex tech stack & environment. So in general, I’m actually enjoying not being “the go-to guy” for every little problem that happens with the database.

And you know what else I discovered? I am pretty darn good at what I do. That’s a good feeling!

But I’m still the DBA. (Technically, one of two, but the other DBA is the company’s old-time-guru-turned-part-time-consultant). So in a sense, being that “go-to guy” hasn’t gone away, it’s just gotten much more manageable – there’s a larger organization of people around me that can help in many different ways. Whether it’s the old-timers with that tribal knowledge of all the hidden rules and nuances of the business, or the SysAdmin who can whip out a PowerShell script in less time than it takes to say “automation”, or the domain expert that can pinpoint exactly why something is broken because they literally “wrote the book” on that problem-space. So it’s really cool.

Plus, it gives me the space and the time to focus on getting really good at DBA stuff – performance tuning, report automation, data integrity, change management, backup/recovery, data security, all that jazz. I’m looking forward to really earning that “Senior DBA” badge – because even though that’s my title, I still recognize that I have a lot to learn. The landscape is constantly evolving – 5 years ago, this whole “Cloud” business was virtually (yes, that is a pun) unheard of; now it’s all anybody talks about. I “grew up” in an on-prem (that’s short for on-premises) Windows + SQL Server + IIS world; but now I’ve got my fingers on AWS instances (both EC2 and RDS), MongoDBs, a SSAS data warehouse, even a sprinkling of MySQL and Azure. Not to mention dozens of SQL Servers spread over 3 offices plus a data-center, and a ridiculously slick converged infrastructure platform called a “VxBlock”. The technology world just keeps getting bigger, better, faster, smarter.

And honestly, that’s why I love what I do.

Written with StackEdit.

Welcome

Picture mid-2006. A fresh college graduate, BS in Computer Engineering. That’s half hardware, half software. Maybe more like 40-60. Not Comp Sci, though looking back, that might have been a little smarter. I learned a bunch about hardware and embedded systems that I will probably never use in my life. But hey, I can probably solder a capacitor to a circuit board without completely frying both it and my fingers!

So I had some education, but not much practical experience… none, really. But luckily enough, I found a job, in my home town, and started my journey! From data-analyst to programmer to database administrator and beyond, it was a wild ride. The technology space is a fascinating, painful, exciting, crushing, inspiring, challenging, awesome field to work in – and yes, those were oxymorons.

Anyway. Enough waxing dramatic for one post. Stay tuned for more about me! (Sure, it sounds self-centered, but isn’t that exactly what a blog is?)

But seriously. Thank you for reading and please do come back to visit. I promise I’ll attempt to entertain, inform, and intrigue. Or at least one of those.

Until next time!

Written with StackEdit.