Tag: twitter

  • Python Command Line Calls

    There was a time I worked in a company any we lived in the command line. This was in the early 90s, prior to Windows v3.1 being released and we primarily used DOS on a Novell network.

    We also had paper phone books on every desk for the 1,000+ people on the property. However, as you might guess, the phone books went out of date, and were only updated a couple times a year. We might get a new sheet, but mostly people learned to cross out a name and write a new extension in for the people they dealt with regularly.

    However updates to the master list happened regularly, every few days. These were done in a small Lotus 1-2-3 file that an administrative assistant maintained. As a network person, I knew where this lived, and arranged a text export of this every night with a scheduled task.

    Why text? Well, I’d come from a university where we had our phone details in a text file and would grep the file to find information. In DOS, I knew we could do the same thing with FIND. However, rather than write the find command with parameters, I put a batch file on our global share that called FIND with the parameters needed and the path to the phone book. I called this 411.bat. When I needed a phone number, I could type

    411 “Andy Warren”

    I’d get Andy’s name, location, and phone number back. It was a trivial piece of programming for me, but the rest of the network team, all non-programmers, were thrilled. I even added a /? check to return help information to the user.

    With my playing with Python last week, I decided to do this for myself as well. I took my Python program to send tweets and changed it to send a tweet when the program was called, and to send the parameter as the tweet. The code looked like this::

    import sys
    import tweepy
    def send_a_tweet(tweettext):
    consumer_key = "X1GWqgKpPP4OuqWJZ7hw6"
    consumer_secret = "QW3EkMHlyzF69AHn8DjyWyRG5CAQ0wjK9RqUZ2"
    access_token = '146009-MqpeZFslj0VGgTSik1fq5klvJpqc1x6HAsiu'
    access_token_secret = '8wIZlPbIqXtczc2LOsJgjMP3dCRRw5ajMvkjEspF'
    
    auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
    auth.set_access_token(access_token, access_token_secret)
    
    api = tweepy.API(auth)
    
    api.update_status(status=tweettext)
    
    if __name__ == '__main__':
    send_a_tweet(sys.argv[1])
    

    I placed this in a “\tools” folder that I have in my path. I also added a “tweet.cmd” file in this folder with this code:

    python c:\tools\send_tweet.py %1

    Since Python.exe is in my path as well, I can do this:

    2015-11-06 15_25_19-Command Prompt

    And I’ll see this on my timeline. I guess you’ll all be able to see this as well.

    2015-11-06 15_25_26-Steve Jones (@way0utwest) _ Twitter

     

    Why bother? Well, it was partially playing around. As I have been learning Python, I have mostly been playing in an IDE, solving small problems, but not really doing things useful. I also like the idea of command line tools, since I find them quick. Tweetdeck is big and bloated, and if I wanted to send a tweet from my desk, this is a quick way to do it. I could do a “readtweets” as well, and may.

    However I also learned how to call Python programs with a command line, which is a good step to starting to build more useful programs that I can customize. This is also the start of me being able to schedule a program, and perhaps build more automation into my life with Python.

    Mostly, however, it was just fun.

  • Python and Tweepy

    One of the projects that’s been on my list lately is to programmatically access Twitter for a few ideas I want to play with. Since I’ve been trying to learn some Python, I thought I would take a look using Python to update status and read status.

    A quick Google search showed me lots of Python clients, but Tweepy caught my eye for some reason. I’m not sure why, but I ended up popping a command line open and downloading the library.

    From there, I saw a short tutorial over at Python Central. I started by creating an app at Twitter for myself, which was very simple. Once that was done, I had a set of consumer tokens (key and secret), that I could use. Another click got me to the access key and secret. Note, the easy way to do this is over at dev.twitter.com.

    My first attempt was using this sample code.

    import tweepy

    consumer_key = “ABCDE”
    consumer_secret = “12345”
    access_token = ‘asdfasdf’
    access_token_secret = ‘98765’

    auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
    auth.set_access_token(access_token, access_token_secret)

    api = tweepy.API(auth)

    public_tweets = api.user_timeline()
    for tweet in public_tweets:
    print(tweet.text)

    This gives me a list of my tweets. At least, the last 20.

    2015-11-06 13_30_24-SDTIG - [C__Users_Steve_OneDrive_Documents_Python_SDTIG] - ..._Last_10_twitter_l

    That’s progress.

    I then went to make an update by using this:

    api.update_status('Hello, Pyton')

    However that returned an error:

    2015-11-06 13_50_53-Cortana

    Hmmm, I double checked the API, and this should work, but I’m guessing there’s another issue. I searched, and found there’s a bug. However, I should be using named parameters, so no big deal. Change the code.

    api.update_status(status='Hello, Pyton')

    Now it works.

    2015-11-06 13_53_06-Steve Jones (@way0utwest) _ Twitter

    This is the first step for me to look at building an app that might send some tweets on my behalf, perhaps with the data stored somewhere, like, I don’t know, maybe a database?

  • One Database to Rule Them All

    This is what you build to juggle 6,000 tweets a second. That’s the headline that caught my eye and it’s about the challenges of Twitter and the data that they handle. Twitter definitely has a tough problem, one that few of us have, but perhaps they can help us learn to better deal with our own data from their experiences on an edge case.

    The story is journalistic, not so technical, but it is interesting. Twitter has struggled with a blend of data that is partially crucial and must be consistent now (usernames) and other data that can be a bit out of date (tweets). They also have lots of unstructured data (photo/video) that is combined with more traditional, structured data. They’ve used a few different database platforms to store this data and assemble it with their application. That’s the same things that most of us also do when we deal with many different types of data.

    However Twitter is trying to find away around dealing with disparate systems. They’ve had a number of engineers working on Manhattan, their database designed to handle both structured and unstructured data. And because they work for Twitter, this platform is designed to manage all of this data with very high workload demands at scale.

    It will be interesting to see if they come up with any innovative ideas. Certainly SQL Server already has options for managing structured and unstructured data, though perhaps not at the scale Twitter needs.

    Steve Jones