Voice of the DBA

Tag: software development

  • Useless Hackathons

    I participated in a few coding competitions in high school, where we got together for a day with a team and tried to build software to solve a problem. They were challenging and exciting, but exhausting. Not something I’d want to do often.

    A number of corporations have tried something similar with internal hackathons as a way to find internal innovation and inspire employees. Getting a day, or a few days, to build some software that might change an organization can be exciting, but the events are not without some drawbacks.

    I’ve liked the idea of us doing this at Redgate, with our Down Tools Week, but the CIO of TD Ameritrade finds hackathons useless. The reasons he gives makes sense, and they are problems I’ve seen in various organizations when employees want to build something new, without necessarily understanding the problem or having the resources to complete their work. I mostly agree that unbounded hackathons are problematic, and having some evaluation of ideas is important.

    It’s hard to be innovative constantly in an organization when you have pressures and deadlines to complete work. At the same time, it can be easy for an entire development department to stagnate because writing software to close out tickets someone else has submitted. That’s boring, uninteresting work in many cases, and it can reduce incentives to build great software. The ideas in the TD Ameritrade piece are good, allowing a team to be sure developers aren’t solving a problem that’s been evaluated, or choosing a solution that is impractical. I like the idea of funding ideas and increasing that over time if ideas show potential.

    Ultimately there are often lots of ideas on how to solve a problem with software, but we rarely work in a vacuum. Our solutions have to fit within an existing structure and have a ROI that makes sense to the organization. Whether we sell software or support some other business, we are trying to achieve those goals, not just solve the problem in the most satisfying manner. Too often I think software developers sometimes forget there are goals other than just producing code that completes a task. Usually those goals are furthering the organization in some way beyond the software.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Should You Learn R or Python?

    I’ve been working on various skills over the last few years, trying to solve some simple problems in Python and PowerShell, in addition to T-SQL, to continue to improve my skills. It’s an interesting challenge at times, especially when I need to use new features or functions to which I haven’t had exposure in the past.

    I also work with R lightly, as I need to build some Questions of the Day for SQLServerCentral and I try to alternate Python and R questions every week. This has caused me to dig in and try to learn more about the language and how to manipulate data.

    Recently I was reading an essay from a consultant that works with clients using both R and Python. The piece talks about the differences and how these work to solve business problems. If you don’t want to read the entire thing, the comparison starts with the simple “you need both”, though there is more to the story.

    The most interesting part of this for me was that the author notes that while these are good languages in different ways for data analysis, they aren’t great for data preparation and SQL is still required. Either a database like SQL Server or a platform like Apache Spark. Part of the reason is that R and Python aren’t very efficient, and as we work with more data and larger workloads, efficiency matters.

    The other part of the piece I liked was the note that we need to collaborate and our work needs to be reproducible for others. I love having git for moving code around and keeping configuration files in a repository of some sort. It has certainly helped me take advantage of bits that others have written and easily reproduce their work on my system.

    While some of us work with just SQL, I expect that we will get involved with other parts of projects and may need to help troubleshoot or improve code. I find both of these languages interesting and a nice complement to each other. I’ve also learned there are places where I much prefer one over the other, especially with some of the Advent of Code problems. Some are simple in SQL, but others are much more suited to Python. I haven’t tried them in R, but I bet some of them would be well suited to that environment.

    If you have tried either, or have a preference, let us know. What are the advantages or disadvantages of each when you are working in a business?

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Open APIs

    The US Supreme Court ruling in Oracle v Google was released recently, with Google winning and removing, or reducing, copyright protection for APIs. That means if you use the same method names, parameter names, etc. as some other software, it is unlikely that the vendor for that software can sue you or force you to change. I think that makes sense, as in many cases, someone might accidentally copy an API in something like PowerShell. Already I’ve seen companies and individuals build “Get-ClusterName” calls.

    I don’t know all the ins and outs of this case. It’s complex, and I’ve seen some notes that Google had permission from Sun Microsystems to use Java. After Oracle bought Sun, apparently things changed, but I’m not sure how this agreement was structured or the legalities of how things evolved.

    In any case, I’m glad Google won. I think the specification of an API isn’t something that should be protected and limited. If a company implements a list of method names and parameters, I don’t feel those deserve protection. The actual code that runs should be protected, but the names in the interface shouldn’t be. This might mean that if I interfaced with, say a piece of retail software that had “GetOrder”, “NewOrder”, etc., and I wanted to switch from vendor A to vendor B, I could without modifying my app.

    This prevents lock-in, which isn’t something I want to see enforced in software. In fact, for things like medical software, I’d want the same API for all of them. This is something that I think will encourage interaction of software, and likely reduce costs for many types of COTS systems that allow connections from PowerShell, bash, and other scripting systems. I think this also helps developers that might work across platforms or systems and could use the same code structures to switch from one to the other.

    This also means that vendors implementing APIs need to do a good job with their software. They can’t hold customers hostage with their system over the cost of reworking code the connects with their software. I think that pressure to be better, whether in features, security, or maybe in all ways, is important for software to evolve and improve.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.

  • Disclosing the How

    Many of us have written a computer program of some sort. Maybe just a script to complete a task, maybe complex software the clients interact with. For all of us, I would guess that we’ve been asked to do something, or we’ve asked ourselves to build a program, and we use that request as a specification for our code.

    When we do this, how many of us get this right the first time? I’m sure some do, and the simpler or narrower the task, the more likely we get it right. However, we also sometimes make mistakes and realize that our specification wasn’t correct or that we interpreted the specification incorrectly. Even when I write code for myself, sometimes I realize I haven’t described when I need clearly enough, even in my own head.

    That happens when I control the entire process. When someone else is my client, there are plenty of ways in which we can make mistakes, especially in decoding the specification. As we use more and more software to make decisions for us and automate our world, I wonder how often we’ve assumed the software works in a way different than it actually does. With machine learning and artificial intelligence programs, I think there is an even greater chance of a misunderstanding.

    Recently I saw that in New York City there is a proposal to disclose how software is used in hiring decisions. Humans are barred from discriminating by law, but what about software? What if there are bias that can come out in the software based on how someone enters data?

    I think there are certainly potential issues in having software evaluate individuals, though I do think software will do a better job than many people, once we’ve worked out the bugs and we know how we want it to work, and then we can compare that to how the algorithm produces results.

    These are tough issues, and difficult projects. I also think it’s important that we analyze the data that is used in testing software, to be sure that it doesn’t contain its own biases. Ultimately I think having public data sets that are evaluated by many individuals is important. I could see a reference set of data and the expected results, or some range of results being required to ensure software meets the goals we want as a society. This, along with the disclosure of how algorithms work will make our use of software better over time.

    Steve Jones

    Listen to the podcast at Libsyn, Stitcher, Spotify, or iTunes.