Tag: AI

It’s interesting to look at the data business and how companies view DBAs, database developers, BI developers, data scientists, and more. There are some companies that really see value in our services, and I’m grateful for that. I’ve been gainfully employed for well over two decades to work with data. There isn’t much standardization in our jobs or what we’re expected to do, but I’ve grown comfortable with that. That’s one of the reasons the people working with me, my coworkers, are more important than the work. The work is the work.

As our systems become more advanced, there is concern over how much less some of our skills might be needed with new AI type systems. Will the move to smarter systems mean there will be more opportunity for us? Or less? Certainly in the long term there might be less jobs if systems become really capable, but in the short term I don’t think so. As much as Microsoft has improved SQL Server, and they’ve done great things with easier HA configs, the Query Store, Adaptive Query Processing (coming) and more, they aren’t replacing many of us. Maybe a few, but I think there’s still lots of technical work.

Darmesh Shah wrote a nice piece on AI and how it will help many of us in our jobs, providing the easy information and guidance for us to focus our skills. He sees bots as helpers, which to me presents new opportunities to interact with and work with customers and data. We will find ourselves more capable with help from AI, not replaced. As we get better machine learning or other adaptive algorithms, we’ll actually find new ways to work with data. This should provide us with new opportunities and new types of jobs that we might grow into. Plenty of people want to dismiss the data science jobs as a popular area where anyone can claim those skills if they know a statistical function and can query data in R, but there are real jobs in those areas, and there are opportunities in new companies that might not have existed in years past.

There are going to be some amazing new ways that data and more intelligent algorithms will help us see our world in the future. There will also be scary ones, and many we don’t know if we can trust. Somewhere in there, the world will become very, very interesting for those of us that work with data on a daily basis, looking for new ways to extract information from all the bits and bytes that we store. Once again, this is something I do look forward to as a data professional.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 4.6MB) podcast or subscribe to the feed at iTunes and Libsyn.

I was reading about the Microsoft Cognitive Services and their wider release in preview to more developers. There are a few of the many machine learning services that anyone can use to build more intelligence into their applications. The entire Cognitive Services include some interesting sets of APIs that allow us to build new features and even new capabilities that we might never have considered without the power of machine learning to change the behavior of systems over time.

With machine learning, we have more and more ways to analyze data, allowing our systems to actually react to the data and become better at their particular task. I think there are certainly some dangers in how these systems might be hacked, but for many uses, I think ways that these platforms can work with speech, images and more might really alter the way in which we decide to build new applications in the future. The goal, at least according to Microsoft, is to have the AI services help humans accomplish tasks, not replace them. That’s something I hope actually comes true.

I do think that we don’t quite know how these machine learning systems will grow and interact with people over time. They work by analyzing data, and altering behavior based on data, which means that we need to better understand the implications and effects of various algorithms and systems. I hope that more companies and developers spend time experimenting and working with various APIs and services to learn more about them. I’d like to see more projects and proof-of-concept systems.

Some of us will just build things that aren’t useful, or that don’t even work well. That’s OK. We need to experiment, and understand these are experiments. These aren’t guaranteed ways of producing information from data. As long as we get that, or at least understand some of the work we may do for clients will need to be thrown away, that’s fine. I know there’s pressure in many companies to be efficient and just work on things that help the organization move forward. I’m sure some companies will make bad decisions, or even abuse their AI systems. I’m also hopeful that more will realize that some experimentation is necessary if you want to find new ways to create richer, more reactive and customized systems based on the massive amounts of data we collect every day.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 4.0MB) podcast or subscribe to the feed at iTunes and Libsyn.

Machine learning and artificial intelligence seem to be the hot topics these days. From bots that can interact with people to systems that learn and grow as they process more data, it seems that science fiction is becoming reality. At least, in limited ways. Autonomous cars, perhaps the highest profile example of these topics, are advancing and being tested in a few locations around the world, but I think we are a long way from having human controlled and autonomous cars interacting freely at any scale. There are still plenty of issues to work out, and the consequences from mistakes require serious consideration.

I was thinking of this as I read an interesting question: Whose black box do you trust? It’s a look at algorithms and machine learning, and the impact they have on the world around us, despite many of us not understanding how they work. The main examples in the piece are in the area of journalism as it relates to social media (primarily Google and Facebook), but also touches on autonomous vehicles, both autos and planes. The latter was a bit of a shock to me as I assumed humans always handled takeoff and landing, something the author says doesn’t happen at SFO. Some searches around pilot sites seem to note that automated landing is done regularly to test systems, but is used in a minority of cases.

The question is, do we trust the black boxes that run our systems, and really, does it matter? In the piece, Tim O’Reilly says he has four tests for trusting an algorithm:

• the outcome is clear
• success is measurable
• the goals of the creators align with the goals of the consumer
• does the algorithm lead creators and users to better long term decisions.

Those are interesting ways to evaluate a system, though I think the problem is that the last two are a bit nebulous. One of the things that I see more and more as I get older is that the same data or the same facts can lead two different people (or groups) to two different results. Our goals, our interpretation of events, even the weights we place on the various factors in a complex system vary dramatically from person to person. In such a world, can we truly evaluate what the goals of a creator are? Forget about consumers, assume one person building a software system. They will have multiple goals, and do we really think their goals can be easily listed? Or weighted/ranked appropriately? What about when the goals change?

I really think that the black boxes need more disclosure, though I freely admit there isn’t a good way I know of to do this. However, I do know one thing that can be better disclosed: data. We can have more openness and analysis of data from software systems along with some accountability by creators for the impacts of their software. Again, I don’t know how to enforce accountability, especially at scales that encompass millions of consumers and easily cross country borders. That is a problem I think we need to find ways to tackle, at least at some manageable level. Maybe using the 80/20 rule where 80% of consumers and creators find the outcome to be a good one.

The world of technology and software are advancing and growing extremely quickly. Certainly hardware advances, but it seems the last 5-10 years have been more about new and different software applications that fundamentally alter the way humans can interact in social, business, and government situations. Underpinning all the changes is data. New data, more data, and novel ways of working with this data in ways that were unheard of 20 years ago. It’s an amazing time to be a data professional.

Steve Jones