Voice of the DBA

Tag: Database Weekly

  • Analyzing Breached Data

    A few of you out there might be data scientists who profile data regularly. Probably a fair number of you do import/export work and learn to check data values, perhaps with counts, distincts, or other aggregates. I don’t know if the performance tuners out there look at the skew of data or the details of what is in a query that needs improvement. However, all of you are likely familiar with data and trying to query it for some type of meaning.

    One of the largest data breaches occurred with National Public Data. Troy Hunt analyzed the breach as a part of his work with haveIbeenpwned. The piece is an interesting analysis of the data, trying to determine both it’s legitimacy as well as what is actually included in the breach. It’s a fascinating read and I encourage you to look at it not just from the data analysis side, but also to be aware of what data about you is being aggregated and sold by companies.

    The read is interesting as it is a bit of a detective story, digging through data in a folder, which is something I’ve had to do. I’ve had people in previous jobs just dump a bunch of data on me and ask me to load it into a database. Or a table. Often without them knowing what type of data it is, what formats, do files relate to each other? Are there multiple tables worth of data in a file? All questions I’ve had to ask myself (and answer), and similar to what Troy did to analyze the breach.

    Data is very important to many of us, in different ways, but I’m often amazed at how few people actually understand how to organize data and ensure others can track the metadata about their data (what their data represents). I’m guessing this is why every person that gets an extract of data to load into Excel formats it in different ways.

    In many cases, people want the ability to query data, but they prefer to just focus on one table that contains a lot of information. They don’t want to know how to “join” data together. I think this might be the reason we see so many views in databases, and why we have views built on views. Each new client of the database needs their own view structure.

    The world of data is a mess, even inside an organization. Once we start moving data between organizations, it’s truly a mess. We might bemoan all the inefficiencies and work we do to move, change, and re-load data as custom, human ETL machines, but there is one great thing about this tangled web. It provides for steady, secure jobs for many of us with no end of work in sight.

    Steve Jones

  • Protecting Schools

    One very common targets for hackers are schools and universities. The latter may have plenty of resources for staff, but often school systems don’t have that same type of budget. In many ways, it’s sad that hackers would target schools that struggle to educate and help others. The staff often deals with low pay and high stress already, and losing access to computer systems adds to an already tough job.

    There was an interview this week with the director of technology at one district. Johnathan Kim works for the Woodland Hills school district and is a former staffer at the Navy Cyber Defense Operations Command. That’s the type of training that I think few school district employees have. I’ve known a few people who manage technology inside schools, and while they are often smart, capable people, they aren’t security professionals.

    The interview talks about a few of the changes that Mr. Kim has made, such as enabling two-factor authentication (2FA) and removing local admin rights for many teachers. I know these are the types of rules that frustrate many workers who use computers. In fact, I ran into someone who rarely upgrades software on their development machine because so many applications require administrative rights and they don’t want to bother opening tickets more than a few times a year.

    Two-factor authentication can be a pain, and I know I get confused sometimes as I have both 2FA and MFA with different processes for different systems. It’s good in that a few times in the last year I’ve caught a hacking attempt, but it’s also a pain to deal with when I’m doing something simple. I can see why people don’t like it when they don’t understand the challenges of securing systems. Every time I find myself frustrated, I stop and remember the problems others have had, especially those that have dealt with ransomware in their organizations.

    A good point in the interview is that education can help smooth the way for security practices that feel unnecessary or disruptive. It’s good to remind ourselves why we want the least privileges needed assigned to others, and why those of us with privileged access need a second account for that access. We also ought to come up with a good story to educate others when they complain, perhaps using a story of a breach or loss to help remind others that our systems are constantly under attack.

    Steve Jones

  • Has AI Changed Software Development?

    I saw a piece that asked the question about AI and software development, specifically ChatGPT, a year after its release. Has software development changed? It’s a question I’ve been trying to ask for myself, though I do think the way AI works for lots of code is fundamentally different than how it works for databases. Databases need a lot of context, with schema and relationships, whereas a lot of other code can be structured in a more narrow sense, focused on an algorithm rather than specifically needing to reference other objects.

    That’s not to say that a generative AI can write application code better than database code. I think in both cases it can be helpful or hurtful. As I read the article, I find myself agreeing that AI has changed software development. It’s a tool, and it can be useful. It can help structure code, point out issues, and help with building tests.

    That being said, I also agree with this sentence in the article: “…given that to write anything resembling maintainable code with AI, you’ll need some level of expertise in programming.” Everything I’ve seen from AIs still requires a review from a human that understands if the AI is on the right track in approaching the problem. I’ve written this a few times (and discussed it with others) that I think an AI produces junior level code in many cases, and it takes a subject matter expert to decide if that’s OK for the situation.

    However, I feel that AI does a few things really well. First, it’s a great search engine, able to understand multiple prompts in a row that narrow in the thing I might spend much more time searching for. It also does some nice predictive things in guessing what code is needed next. In most application languages, this is helpful. In SQL, not so much, since the SQL language lays code out in a way that makes it harder to predict things. The LINQ structure of from, where, select is much more cleaner in many ways. If only RDBMSes natively executed LINQ…

    If you’ve tried AI, what has worked, what prompts give you help? Is there anything especially time-saving you’ve found? I, and many others, are looking to understand where it can help us.

    Steve Jones

  • No Degree Needed

    For most of my career, I’ve seen many companies that hire white-collar workers, including their IT staff, require a college degree or the equivalent experience in the military. Often this has been something the Human Resources department has made a requirement in their job descriptions and requests to recruiters. This was a filter that stopped many talented people from even getting an interview.

    That hasn’t stopped a lot of people from pursuing careers as developers or IT staffers, and I’ve had many friends who have had successful careers without a degree.

    The world has been changing, especially since the pandemic, and many companies are no longer requiring any sort of university degree for candidates. There are still a lot of job descriptions that “prefer” a degree, but even that is changing rapidly. I hope at some point that we stop looking at degrees as anything other than a bit of experience in working through a project and not as any sort of qualification for a particular job.

    This week there was an article about seven ways to become a software engineer without a degree. The ways are: learn to program, earn a cert, contribute to an OSS codebase, write documentation, find employers who don’t care, be a freelance developer, and go to a bootcamp. Of these, a few seem redundant learning to code is probably needed to contribute to OSS or freelancing.

    Of these, I overall like the advice. Being self-motivated or driven is a skill that many employers appreciate. Showing that you are producing something useful in the world is a good way to create opportunities. It’s also a sign that you’ll work to be productive and not expect to avoid work after getting hired. I know I prefer people who go figure things out rather than those who wait for someone else to tell them what to do. I don’t recommend bootcamps, however, unless you are very motivated to take whatever you learn and then expand on it with OSS contributions or other tasks that showcase and expand knowledge.

    Finding a job is challenging at times, and certainly impressing someone enough to hire you is a task. A degree can help, especially with those who might be prejudiced towards university experience, but more and more people recognize that college doesn’t necessarily prepare you to be productive or a great employee. You can prove that to people yourself with some work, some documenting your efforts, and some good soft skills to explain what you know.

    Steve Jones