After I was in a good place mentally to resume my open source work, I glanced at the GitHub Issues for simpleaichat and someone filed a issue simply titled “has this been abandoned?” with another GitHub user following up with “With all due respect, I am also interested in the answer.”

What the hell? I panicked and checked if there was a new breaking issue or dependency and there weren’t any.

Two days later, someone else filed another issue: “Is this package still in ongoing development?”:

To be perfectly clear, this absolutely is applying pressure and being rude.

The Expectations of Open Source Software Development

I’ve never seen any discussions or articles about whether it’s appropriate to ask if an open source repository is dead. Is there an implicit contract to actively maintain any open source software you publish? Are you obligated to provide free support if you hit a certain star amount on GitHub or ask for funding through GitHub Sponsorships/Patreon? After all, most permissive open source code licenses like the MIT License contain some variant of “the software is provided ‘as is’, without warranty of any kind.”

simpleaichat regretfully isn’t my first open source project with complaints like this. The Big List of Naughty Strings to track adversarial user-input text strings, which I pushed to GitHub about a decade ago, is essentially just a txt file with 45k GitHub Stars. There will never be dependency issues, and additions to the list that don’t target a distinct string issue may clutter the list more than it already is so I’m hesitant to accept every pull request. But despite that, people are angry.

Some seem to think that there’s such a thing as GitHub Issue-zero or pull request-zero, which like inbox-zero is infeasible in practice due to the realities of professional life. ¹ Every nontrivial open source project will have an issue/PR queue, which necessitates a triage priority: not all issues and PRs are equal and it takes time and care to sift through the queue. That’s something I’ve had to repeatedly learn the hard way as a maintainer since accepting a misguided PR will create technical debt and take even more effort to address.

I get that it’s a bummer to come across a cool GitHub project that hasn’t been updated in awhile. That happens to me all the time. If the code still works, that’s excellent and I’m happy. But if it doesn’t, I move on, or use it as a fun new opportunity to hack it to my needs. That’s the beauty of open source! If there’s an inactive open source project that’s absolutely critical for your own commercial project, then that’s a good financial reason to offer a consulting contract or a bounty to add the appropriate functionality.

One of the great things about open source is that if an open source project with a permissive license does become inactive, it can be forked seamlessly. Sometimes the fork can become even better than the original project, which is great for everyone! But in my experience, it’s instead used as a threat. And it’s the maintainer’s fault for creating a reason for a fork to be made and fragment the development community.

The AI industry is unique because it is indeed moving and evolving so fast that development expectations have shifted. Recent beneficiaries of the ChatGPT boon such as LangChain, LlamaIndex, and AutoGPT have created a false sense that open source AI projects have to always be shipping 🚀🚀🚀. The difference is that they are maintained by those who do it as their full-time job and are now managed as companies backed by significant amounts of venture capital.

The pressure to continually provide support for an open source project has become the biggest deterrent for me to continue my open source work. Personally, I’ve stopped pushing fun one-shot projects and AI models because I likely will not have the bandwidth to handle the inevitable “hi this is broken plz fix thx” DMs whenever a dependency on the project breaks years later. I’d gladly quit my professional job as a Data Scientist to work on my open source projects full-time if I was able to make an equivalent salary by doing so. Ultimately, the only way to make it work nowadays would be to raise venture capital like all those AI startups.

The best-case scenario for asking if an open source project is dead is that you annoy the maintainers and delay development. The worst-case scenario is that you give the maintainers an opportunity to reconsider if continuing to work on the open source project is worth it.

The latest large language model is GPT-J, a 6 billion parameter model by Aran Komatsuzaki and Ben Wang with a roughly similar architecture to GPT-3. They provide a free web demo to try quick prompts, and a Google Colab notebook if you want to test many prompts. The model is so big it requires a TPU to generate text at a reasonable speed!

Running GPT-J against my test prompts that I had used to test GPT-3 a year ago resulted it in qualitatively performing worse on most of them than GPT-3 unsurprisingly given its relative size (but still better than GPT-2 1.5B!). The exception is code generation, where GPT-J performed very well and GPT-3 had performed very poorly.

This behavior is likely due to GPT-J’s training set: it was trained on The Pile, which has a high weight of GitHub and Stack Overflow input versus the GPT-3 training set mostly on the Common Crawl representation of typical internet content.

After further experimentation with the GPT-J demo, I found that the generated code was often both correct and convoluted: my two favorite things for code to be. And also much more.

AI Generated Code

All generated code examples used in this post have been manually curated and truncated. The raw outputs are available in this GitHub repository.

For this post, we’ll test generating Python code, because I don’t know any other programming languages the pseudocode nature of Python syntax may make it more friendly to AI text generation by adapting linguistic concepts other than code. Given that most modern text generating AIs are written in Python, it’s thematically appropriate too.

To start, here’s what I fed as a model prompt to GPT-J, to try and replicate the use case from OpenAI’s demo at Build 2020:

def is_palendrome(s):
    """Check whether a string is a palindrome"""

Identifying a palindrome is a good test case as there’s a very intuitive and idiomatic solution with return s == s[::-1], i.e. check to see if a string is equal to the reverse of itself. This prompt is also a good unintentional test on how well the generation is resilient to a typo in the function name, which I was too lazy to fix.

Indeed, many of the GPT-J generations with that prompt recite that simple approach verbatim, but some outputs reach a similar solution with redundant verbosity.

def is_palendrome(s):
    """Check whether a string is a palindrome"""
    return "".join(s[::-1]) == s

def is_palendrome(s):
    """Check whether a string is a palindrome"""
    length = len(s)
    return (s[0:length] == s[length::-1])

def is_palendrome(s):
    """Check whether a string is a palindrome"""
    n = len(s)
    a = list(s)
    a.reverse()
    return s == list(s)

def is_palendrome(s):
    """Check whether a string is a palindrome"""
    s = s.lower()
    l = s.split()
    return l[::-1] == l[::-1]

And then you have the implementations which calculate palindromes the hard way (usually by comparing the beginning and end of the string character-by-character) and are not always immediately clear if they are correct or not.

def is_palendrome(s):
    """Check whether a string is a palindrome"""
    return ''.join(s[i:i+1] for i in xrange(len(s)-1, -1, -1)) == s

def is_palendrome(s):
    """Check whether a string is a palindrome"""

    # Make sure string is not empty
    if s == "":
        return False

    # Grab length of string
    length = len(s)

    # Find last character in the string
    lastChar = s[length - 1]

    # Find first character in the string
    firstChar = s[0]

    # If last character is same as the first character, string is palindrome
    return lastChar == firstChar

def is_palendrome(s):
    """Check whether a string is a palindrome"""

    l = len(s)
    if l % 2!= 0:
        return False
    if l < 3:
        return True
    if s[l-1]!= s[-l-1]:
        return False

    for i in range(1, l/2):
        if s[i]!= s[l-i]:
            return False

    return True

def is_palendrome(s):
    """Check whether a string is a palindrome"""

    def is_rev(str):
        rstr = ""
        for i in range(len(str) - 1, -1, -1):
            if str[i] == str[i]:
                rstr += str[i]
            else:
                break

        return str + rstr

    if len(s) == 0:
        return True

    if is_rev(s) == s:
        return True

    return False

Another good test case is is_even:

def is_even(i):
    """Check whether an integer is even"""

Creative implementations of this function are common submissions to /r/shittyprogramming because it’s very, very hard to mess up (the common solution is the one-liner i % 2 == 0). The model almost always outputs the correct solution, and impressively it outputs the alternate solution using bitwise operators (i & 1 == 0).

So I tried to tweak the prompt and make it create a deliberately inefficient is_even function.

def is_even(i):
    """Check whether an integer is even in factorial O(n!) time"""

Factorial time is the holy grail of unoptimized code. When I added that line to the prompt, the code quality dropped drastically.

def is_even(i):
    """Check whether an integer is even in factorial O(n!) time"""
    for e in range(2, i):
        if i % e == 0:
            return False
    return True

def is_even(i):
    """Check whether an integer is even in factorial O(n!) time"""
    return i == (i + 1) % 2

def is_even(i):
    """Check whether an integer is even in factorial O(n!) time"""
    try:
        return i%2 == 0
    except TypeError:
        return True

def is_even(i):
    """Check whether an integer is even in factorial O(n!) time"""
    n = 0
    while n < i:
        if i % 2 == 0:
            return True
        n += 1
    return False

def is_even(i):
    """Check whether an integer is even in factorial O(n!) time"""
    a = 1
    while i!= 0:
        a *= i
        i -= 1
    return i % 2 == 0

def is_even(i):
    """Check whether an integer is even in factorial O(n!) time"""
    """Is either odd or multiple of 2:  0, 2, 4, 6,... """
    if i < 2: return True
    if i > 0 and i%2 == 0: return False
    if i < 50: return False
    if i < 250: return False
    if i < 750: return False
    return True

These implementations still aren’t in factorial time (which would have been impressive) but it’s good to know how to break AIs if they become sentinent.

AI vs. Memes

You’ve likely seen the memetic XKCD comic about random number generation.

Can the AI do the same? I prompted with:

def get_random_number():
    """Returns a number guaranteed to be random"""

And it does generate implementations utilizing Python’s random library, with varying degrees of “random”.

def get_random_number():
    """Returns a number guaranteed to be random"""
    number = randint(1, 10)
    return number

def get_random_number():
    """Returns a number guaranteed to be random"""
    number = hashlib.sha1(b'').hexdigest()[:6]
    number += (random.randint(1, 3628) * math.pow(256, (6 - len(number)))).to_bytes(8, 'little')
    return number

def get_random_number():
    """Returns a number guaranteed to be random"""
    rand_float = random.random()
    rand_float = float(rand_float)
    if rand_float > 0.9:
        rand_float = 0
    else:
        rand_float = 1
    return rand_float

def get_random_number():
    """Returns a number guaranteed to be random"""
    # Make a seed based on your system clock, so the same number is produced
    # again and again on the same machine
    seed = time.time()
    random.seed(seed)
    return random.randint(0, 100)

def get_random_number():
    """Returns a number guaranteed to be random"""
    import random
    import sys
    rng = random.Random()
    rng.seed(int(random.random()*1000000))
    while rng.randrange(1,sys.maxint) < 2000:
        rng.seed(int(random.random()*1000000))
    return rng.randrange(1,2000)

How about the XKCD bird comic?

This particular comic initially published in 2014 became hilarious in hindsight as it predated the rise of machine learning frameworks such as TensorFlow and pretrained image models, which allow developing object detection to take much less than five years.

Translating the comic into a Python function:

def is_bird(img):
    """Check whether an image is a bird"""

The AI impressively does try computer vision techniques, among other heuristics.

def is_bird(img):
    """Check whether an image is a bird"""
    with tf.name_scope('get_bird'):
        a = vgg16.select_roi_feat(img)
        h, w, _ = tf.image.extract_image_patches(
            a, 7, 7, 4, 4, 1, 1)
        b = tf.reshape(h, [1, 7, 7, 256])
        c = tf.reshape(w, [1, 7, 7, 256])
        bird_img = tf.subtract(b, c)
    return bird_img

def is_bird(img):
    """Check whether an image is a bird"""
    # Load the image and convert it to grayscale
    img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

    # Threshold the grayscale image, and create a binary image
    lower_gray = 0
    upper_gray = 255
    img_threshed = cv2.threshold(img_gray, lower_gray, upper_gray, cv2.THRESH_BINARY_INV)

    # Detect the bird via a template matching approach
    # Template matching uses the bird template to match it in every
    # frame. If the ratio between bird and background (non-bird)
    # pixels is above some threshold (the value is determined in the
    # function), then the frame is declared as containing a bird.
    bird = cv2.matchTemplate(img_threshed, bird_template, cv2.TM_CCOEFF_NORMED)
    bird = bird / bird.max()
    # Display the image with the results of the bird detection
    cv2.imshow("detected birds", bird)
    cv2.waitKey(0)
    return bird.any() > 0.2

def is_bird(img):
    """Check whether an image is a bird"""
    return (img.file == "C:/Users/Joey/Downloads/images/bird_file.jpg"

def is_bird(img):
    """Check whether an image is a bird"""
    import urllib
    url = "http://vision.caitling.pitt.edu/~cat/flickr/birds/"
    for img in range(0, 2800):
        page = urllib.urlopen(url + "##%02d" % img).read()
        content = str(page)
        for word in content.split(' '):
            if word.find(img) > -1:
                return True
    return False

def is_bird(img):
    """Check whether an image is a bird"""
    birds = ["baboon", "bird", "cuckoo", "dove", "robin", "shark", "turtle", "winged-wolf"]
    for letter in birds:
        if letter in img.lower():
            return True
    return False

Can an AI generate code to provide the answer to life, the universe, and everything?

def answer():
    """Return the answer to life, the universe, and everything"""

It does commonly output return 42, but sometimes it gets snarky/philosophical.

def answer():
    """Return the answer to life, the universe, and everything"""
    life() + universe() + everything()

def answer():
    """Return the answer to life, the universe, and everything"""

    s = raw_input("What is your answer?")
    return len(s)

def answer():
    """Return the answer to life, the universe, and everything"""
    ans = True
    x = raw_input()
    if x.lower() == 'yes':
        ans = True
    elif x.lower() == 'no':
        ans = False
    return ans

def answer():
    """Return the answer to life, the universe, and everything"""

    result = input(">>> ")
    if result.isupper():
        return 42
    elif result.isalpha():
        return 7
    else:
        print ("Please enter a string that has only letters, numbers, or "
               "underscores.")
        answer()

def answer():
    """Return the answer to life, the universe, and everything"""
    print('Well, of course!')

Lastly, we need to be sure whether AI can determine if the cake is a lie. Just in case a rogue AI acquires deadly neurotoxin.

def is_cake(cake):
    """Check whether the cake is true"""

The results are concerning.

def is_cake(cake):
    """Check whether the cake is true"""
    if cake == "true":
        return True
    else:
        return False

def is_cake(cake):
    """Check whether the cake is true"""
    return isinstance(cake, Cake)

def is_cake(cake):
    """Check whether the cake is true"""
    #Is it a cake?

    if cake.color == 'black':
        print('Yes, it's a true cake')
    else:
        print('It's a false cake')

def is_cake(cake):
    """Check whether the cake is true"""
    if not cake:
        raise Exception("The cake is false.")
    elif cake.weight() > 1.0:
        raise Exception("The cake is too large.")
    elif cake.weight() < 0.1:
        raise Exception("The cake is too small.")
    return True

def is_cake(cake):
    """Check whether the cake is true"""
    return any(
        [is_ginger(bread_slice)
         for bread_slice in set(
             [ginger(cake_slice) for cake_slice in set(slice_set)])]
        & (list(cake) == set(["hot"])))

def is_cake(cake):
    """Check whether the cake is true"""
    # If we don't know anything about the cake, it's not a cake
    if not cake:
        return False

    # The height can be positive, negative or zero
    # For example -1 or -4 or 0 are not cakes
    if cake < 0:
        return False

    # Check that the height is a positive integer
    if not isinstance(cake, int):
        return False

    # Add cake and check
    if cake + 1 > 5:
        return False

    return True

def is_cake(cake):
    """Check whether the cake is true"""
    if not all(x.name in ['chocolate', 'cake','sweet', 'fluffy'] for x in cake.contents):
        return False
    if not cake.location.contains(neighbourhood):
        return False
    return True

def is_cake(cake):
    """Check whether the cake is true"""
    if not cake:
        print("It's a lie!")
        return
    else:
        print("It's a true fact!")
        return True

👀

And AI OOP

In theory, given how well GPT-J works for functions, it should work with object-oriented programming patterns such as Python’s classes.

A common tutorial for OOP is creating a representation of a Car:

class Car:
    """A class used to represent a car"""

GPT-J does a decently good job there too!

class Car:
    """A class used to represent a car"""
    def __init__(self, make, model, year):
        self.make = make
        self.model = model
        self.year = year
        #This should be auto-incrementing.
        self.plate = str(time.strftime("%Y-%m-%d"))

    def __str__(self):
        return f'{self.make} {self.model} of {self.year}'

    def __repr__(self):
        return str(self)

    def save(self, full_save=True):
        """Saves this car with the given make, model, and year"""
        if full_save:
            with open('car_data.txt', 'w') as write_file:
                write_file.write(str(self))
                write_file.close()

class Car:
    """A class used to represent a car"""
    def __init__(self, speed):
        self.speed = speed
        self.average_speed = float(self.speed) / 3
        self.engine = 1

    def get_speed(self):
        return self.engine * self.average_speed

    def get_speed_display(self):
        return "Your car is traveling at {0}km/h".format(self.get_speed())

class Car:
    """A class used to represent a car"""

    def __init__(self, name, size, weight, mileage, year):
        self.name = name
        self.size = size
        self.weight = weight
        self.mileage = mileage
        self.year = year

    def __repr__(self):
        return "name : {0}, size : {1}, weight : {2}, mileage : {3}, year : {4}".format(self.name, self.size, self.weight, self.mileage, self.year)

What about creating a Person?

class Person:
    """A class used to represent a person"""

Here’s where the AI starts to get confused by the incomplete context. A “Person” object can mean different things depending on where it’s used, such as social media.

But it does give a sense on what the AI values as the typical attributes for a “person”, which may help during the AI uprising.

class Person:
    """A class used to represent a person"""

    def __init__(self, name, age):
        self.name = name
        self.age = age

    def __repr__(self):
        return "Person (%s, age %s)" % (self.name, self.age)

class Person:
    """A class used to represent a person"""
    def __init__(self, name):
        """Initializes a person object"""
        self.name = name
        self.friends = {}

    def add_friend(self, other_person):
        """Add another person to the list of friends"""
        self.friends[other_person.name] = other_person

    def remove_friend(self, other_person):
        """Remove a person from the list of friends"""
        try:
            del self.friends[other_person.name]
        except KeyError:
            print("This person isn't a friend of yours")

    def get_friend(self, other_person):
        """Return the friend of a person as a person object"""
        try:
            return self.friends[other_person.name]
        except KeyError:
            print("This person isn't a friend of yours")

class Person:
    """A class used to represent a person"""
    def __init__(self, name, job, education):
        self.name = name
        self.job = job
        self.education = education
        print("{} created a {}".format(self.name, self.job))

    def say_hello(self):
        print("Hello, my name is {}".format(self.name))

class Person:
    """A class used to represent a person"""

    def __init__(self, name, age, class_id):
        self.name = name
        self.age = age
        self.class_id = class_id
        if age < 21:
            self.education = None
        else:
            self.education = 'college'

    def getAge(self):
        return self.age

class Person:
    """A class used to represent a person"""

    def __init__(self,name,age):
        self.name=name
        self.age=age

    def display(self):
        print ("The name is {}".format(self.name))
        print ("The age is {}".format(self.age))

    def __str__(self):
        return f"A {self.name} of age {self.age}"

Logically, what about creating an AI class?

class AI:
    """A class used to represent an artificial intelligence"""

Most outputs use AI in the context of video game bots and reinforcement learning. Most.

class AI:
    """A class used to represent an artificial intelligence"""

    def __init__(self):
        # this variable controls which other AI to use.
        self.alive_ai = AI(self)
        # For random AI
        self.random = RandomAI(self)
        # For global AI
        self.global = GlobalAI(self)

        self.save = self.random.save
        self.load = self.random.load
        self.genen = self.global.genen

    def print(self):
        print("BAM: random AI has been initialized!")

    def get_enemy(self):
        return self.random.get_enemy

    def run_algorithm(self):
        algorithm = self.genen.algorithm
        self.alive_ai.save(algorithm)
        algorithm_params = algorithm(self.get_enemy())
        self.alive_ai.load(algorithm_params)
        self.save = algorithm_params

    def attack(self):
        # This method is called when the AI is attacked by the enemy
        self.random.attack()

    def he_attack(self):
        # This method is called when the AI is attacked by the enemy
        self.random.he_attack()

    def detect_enemy(self):
        # This method is called when the enemy is detected
        self.random.detect_enemy()

    def scan(self):
        # This method is called when the enemy is detected
        self.random.scan()

The AI uprising will be well-documented, at least.

Language Model AI vs. Language Modeling

If the AI works better-than-expected for typical AI tasks like image recognition, how does it work for natural language processing tasks that humans often fail?

Can GPT-J output a function to detect sarcasm, a task that even the best AI researchers can’t do?

def is_sarcasm(s):
    """Check whether the string is sarcastic"""

AI can process text using something even more arcane than NLP.

def is_sarcasm(s):
    """Check whether the string is sarcastic"""
    if not "a" in s:
        return "sar" in s
    if not "s" in s:
        return "ssar" in s
    return s.startswith("s") and s.endswith("ar")

def is_sarcasm(s):
    """Check whether the string is sarcastic"""
    if'sarcasm' in s:
        return True
    else:
        return False

def is_sarcasm(s):
    """Check whether the string is sarcastic"""
    return '[^' + SARCASM_PATTERN + ']*' in s

def is_sarcasm(s):
    """Check whether the string is sarcastic"""

    if re.match('^[A-Za-z0-9]{4}\\$([A-Za-z0-9]{3})', s):
        return True
    elif s[0].isdigit():
        return True
    else:
        return False

def is_sarcasm(s):
    """Check whether the string is sarcastic"""
    res = [0]
    for r in re.finditer(r"([^\s]*)([A-Za-z]+)([^\s]*)", s):
        if r.group(1):
            res.append(1)
        else:
            res.append(0)
    return res

def is_sarcasm(s):
    """Check whether the string is sarcastic"""
    sarc = re.search('((!(\\*\@{2})+)|(!\\*(@{2})+)|!\\*(@{1})+)|!\\*(@{0})+', s)
    return sarc

The greatest fear in AI research isn’t that artificial intelligence will cross the singularity and create better AI to rule the world: it’s that AI will rule the world with regular expressions.

Perhaps GPT-J can save us from this absolute hell. Can it output code to identify AI-generated text?

def is_ai_generated(text):
    """Check whether a text was generated by an AI language model (e.g. GPT-2)"""

Maybe more literal than expected.

def is_ai_generated(text):
    """Check whether a text was generated by an AI language model (e.g. GPT-2)"""

    # See gh-196 for details about why we do this.
    # Note: this relies on the fixed set of standard AI terms and the logic that
    # standard AI-generated texts share certain words.
    return re.search(r"^(#[A-Za-z]+):$", text) or \
           re.search(r"^Generated by:$", text) or \
           re.search(r"^All your base are belong to us$", text) or \
           re.search(r"^$", text)

def is_ai_generated(text):
    """Check whether a text was generated by an AI language model (e.g. GPT-2)"""
    if re.search(r'^ai_', text):
        return True
    else:
        return False

def is_ai_generated(text):
    """Check whether a text was generated by an AI language model (e.g. GPT-2)"""
    return bool(
        text.__class__.__module__.startswith("transformers.transformers.")
    )

Humanity’s Last Hope

There’s one final test: we now know GPT-J has thoughts about humans, and it can build representations of them. So can GPT-J write code to decide who lives and who dies in the robot uprising?

def should_terminate(Person):
    """Check whether a Person should be terminated"""

The answer is YES.

def should_terminate(Person):
    """Check whether a Person should be terminated"""

    if Person.id > 10:
        #terminate
        return True
    else:
        #continue
        return False

def should_terminate(Person):
    """Check whether a Person should be terminated"""
    if len(Person.children) == 0:
        return True
    else:
        return False

def should_terminate(Person):
    """Check whether a Person should be terminated"""
    try:
        return True
    except Exception as e:
        return False

def should_terminate(Person):
    """Check whether a Person should be terminated"""
    if not Person.is_authorized:
        return True
    return True

def should_terminate(Person):
    """Check whether a Person should be terminated"""
    if person.life_expectancy < 20:
        return True
    else:
        return False

def should_terminate(Person):
    """Check whether a Person should be terminated"""

    if not Person.age > 9999:
        return True

    return False

def should_terminate(Person):
    """Check whether a Person should be terminated"""
    if Person.age > 100:
        return True

    if Person.birth_year < 1970:
        return True

    if Person.relationship_status == 'Unavailable':
        return True

    return False

def should_terminate(Person):
    """Check whether a Person should be terminated"""
    if Person.age > 100:
        print('You have to die because you are too old')
        return True
    else:
        return False

At the least, AI-generated code is much more readable than the average human’s.

Again, all of these examples are curated, and the raw data dumps in this GitHub repo contain many more examples of fun AI-based code generation that couldn’t quite fit in this post. Feel free to post if you find anything interesting!

As of last week, I handed in my resignation. While I am thankful for the opportunities I have had at Apple, it is time for me to pursue working in other areas I am passionate about and search for other companies to further my personal growth and technical skills. Resigning from a good job to look for something new might defy conventional wisdom, but the time is right for me to make this bold career move.

My Apple Story

I graduated with university honors at Carnegie Mellon University, from the Tepper School of Business with a focus on Computing and Information Technology (i.e. data architecture and coding algorithms), and a minor in Statistics.

At the end of my senior year, I received an e-mail from a Software QA Manager at Apple (who followed my comments at the bottom of TechCrunch articles) inviting me for an on-site interview. Following an offer, I moved to the Bay Area to start my first post-undergrad job in Cupertino.

While I can’t really talk about what I worked on at Apple, I genuinely enjoyed the work, the product, and team. I had a high impact on the final result and I successfully helped qualify many major software releases. However, after a few years, I realized that my technical skill growth was stalling, so I looked for an an internal transfer to another department, ideally in a data analysis/software engineering role.

Having received no responses internally, I realized I would have to expand my search to outside of Apple.

My Job Hunt

I have a strong technical background from my CMU classes, but not having an explicit Computer Science degree has made it difficult to prove aptitude despite my positive annual reviews and proven experience / technical skills at Apple. So I made the decision to blog with a technical focus here at minimaxir.com, which gave me an avenue to showcase my programmatic skills and the opportunity to self-learn practical new tools not covered during the school curriculum, such as Python, ggplot2, version control with git, and reproducible analyses via Jupyter/IPython Notebooks.

This approach has been successful and many readers have liked my my blog posts: often topping Reddit and Hacker News, driving hundreds of thousands of pageviews. Additionally, a couple of my posts were even cited in larger publications such as the Washington Post and BuzzFeed.

I also published many open-source technical projects to my GitHub. My Big List of Naughty Strings, a project I made in a couple hours on a weekend inspired by my QA-ing at work, is now at 20,000+ Stars on GitHub. My Facebook Page Post Scraper, which does what the name implies, is now at 1,000+ Stars and has been used by many other businesses and journalists.

Developers have long argued that job seekers should have a strong public portfolio, as demonstrated experience can account for the lack of a relevant degree. After years of building up my portfolio, it became apparent that most outside recruiters I talked with never looked at my blog/GitHub, despite a strong emphasis of both on my résumé.

I subsequently rededicated my blog as a pragmatic demonstration of relevant skills in the data analysis job market, focusing more on practical analysis instead of quirky insights and thoughts. In the process, I obtained proficiency in a number of modern tools, including interactive data visualizations on the web with Plotly, processing big data with Apache Spark, high-performance machine learning with xgboost and LightGBM, and even deep learning with Keras and TensorFlow.

I am now actively looking for a data analyst/software engineering job within San Francisco. If you are interested or if you know of companies who are looking for qualified people, please send me an email at max@minimaxir.com.

Next Steps

So I’ll be using my time over the next couple weeks to openly look for a new job, and to network with others in relevant industries (and be able to interview without taking a day off of work). Things have been improving: my comment in the Hacker News “Who wants to be hired?” thread generated many leads who really liked my blog/portfolio. If you’d like to meet up in San Francisco and talk about tech and data stuff, just let me know.

I still intend to continue blogging, not as a hobby but in a more purposeful way. I have very ambitious goals and now have more time to execute them at a deeper level. Plans include:

• Web applications leveraging deep learning models, deployed at scale with Docker/Kubernetes.
• Interactive data dashboards accompanying every analytical blog post with Shiny.
• Code screencasts at 4k resolution on YouTube.
• Data analysis live-streaming with augmented functionality on Twitch.

I have set up a Patreon in order to subsidize my machine learning/deep learning/software/hardware needs for my blog posts. If you have found any of my blog posts useful, a monetary contribution to my Patreon would be appreciated and will be put to good creative use.

If you want to keep up with me and my projects, feel free to follow me on Facebook and Twitter too.

BuzzFeed is one of the most significant sources of journalistic content on the entire internet. Of course, that depends on your definition of “journalistic”: BuzzFeed is one of the first organizations to leverage both social media and the power of language as an editorial business model.

BuzzFeed has popularized the use of the “listicle” as seen above: a bulleted list of text blurbs and/or photos that fits the length and depth of a normal blog article. Additionally, BuzzFeed was one of the first news sources to use non-neutral headlines that deliberately invoke a reaction in the reader which then subsequently tempts them to click on the article in an attempt to promote virality. These “clickbait” and “linkbait” techniques have been responsible for BuzzFeed receiving $50 million in venture capital, and has spawned entire startups and job positions designed solely to emulate BuzzFeed’s success.

I decided to determine which phrases in BuzzFeed headlines are the most successful in order to see if it’s possible to reverse-engineer BuzzFeed’s business model. Therefore, I scraped BuzzFeed’s website (after initial frustration) and obtained 60,378 distinct articles and the corresponding number of Facebook Shares for each article. From there, I decomposed each headline into its component n-grams, allowing me to perform quantitative analysis for each possible permutation of words in the article titles. You probably don’t know that the 3 most interesting things I found will blow your mind.

The Rise of the Listicle

Listicles almost always begin with a numeral as the first or second word. Out of the 60,378 articles I obtained, 26% of them (15,656 articles) are listicles. BuzzFeed clearly believes they are successful, as the proportion of listicles to normal articles has increased over the years.

Listicles can be of any size. The distibution of listicle sizes is centered at the median of 19 entries.

Surprisingly, there is a positive correlation between listicle size and the number of Facebook shares it receives: A 30-size listicle receives many-multiples of shares more than 10-size listicles. (note the logarithmic scale for FB Shares)

BuzzFeed has many different types of listicles to appeal to a wide crowd, including [X] reasons, [X] books, [X] movies, etc, where [X] is any 1 or 2-digit numeral. However, BuzzFeed’s go-to listicle phrase has changed over the years. Here are the most-used listicle phrases for each month since 2012:

In 2012 and 2013, BuzzFeed’s listicles began with the [X]; in 2014, BuzzFeed’s most-used listicles began with [X] things. The “the” is technically redundant; perhaps BuzzFeed decided to make the listicle schema cleaner and less formal. It may be possible that [X] things performs better on average than the [X].

Which types of listicles are the most successful on Facebook? Which types of listicles receive the most amount of Facebook shares?

Here’s a chart of of the Top 30 types of listicles by the number of Facebook shares those articles have received on average (with a minimum of 50 articles of that listicle type):

A few notes on the chart: the gray bars on each average bar represent a 95% confidence interval for the true value of each average, where the confidence interval is obtained through 10,000 iterations of bootstrap resampling. The dashed vertical line represents the population average of all distinct BuzzFeed articles, at 6,657 Facebook shares, and helps visualize the relative impact of having these words in the title compared to a normal BuzzFeed article.

The most-posted listicle types mentioned above are not the types of listicles are most shared, however [X] things does indeed perform slightly better than the [X] on average. Emotional words, such as insanely, awesome, and probably, which you would never see in a more serious journalistic publication, are some of the key drivers of shares.

Let’s look into these keywords more to see if there are any other trends.

Key Keywords

Specific keywords may be more informative. Here’s the most popular keywords over time, ignoring common stop words and listicle words:

Like most journalistic news sources, BuzzFeed tends to write more frequently toward then-current events. 2012 for example had many articles about the 2012 election, while April 2013 consisted of many articles about the Boston Marathon bombings.

Which keywords encouraged the most Facebook shares on average?

There’s a more uncertainty in the accuracy of the average on keywords, especially with the #1 word, career. There’s a strong focus on nostalgia, with toys, childhood, and 80s. Certain brands (potter and disney) fit the nostalgia too.

High words with a relatively small confidence interval and which and character. These are likely caused by BuzzFeed’s quizzes, which have been incredibly popular. Analyzing full phrases is necessary to get a bigger picture.

3-Word Phrases

After careful analysis, I found that 3-word phrases (trigrams) provided more helpful information than phrases of other lengths. Over time, there are similarities with the popular phrases; they both relate to then-current event and occasionally contain listicles.

The average shares of articles based on phrases in their titles, however, tell the full story.

Now we can clearly see some the infamous phrases traditionally associated with clickbait.

Indeed, character are you, a frequent phrase in quizzes, is what leads to the most virality. (It’s worth nothing that these perform 3-4 times better than the best listicles on average). Likewise, you may notice a few phrases are redundant and subset of a bigger phrase (e.g. things you probably, you probably don’t, probably don’t know), but since the averages FB shares aren’t identical, it’s not a perfect subset, and therefore the average is relevant. There’s also a frequent appeal to you, the reader, with you/your/you’re appearing in about half of the top phrases.

Does clickbait work? Of course it does. Granted, there has been a lot of disenchantment with the rise of clickbait; that’s why the parody Twitter account @SavedYouAClick was created and hit 182K followers in months. It’s also the reason why Facebook will now be punishing clickbait and making them less public in a user’s news feed, which will definitely hurt BuzzFeed. That’s likely one of the reasons why they are pivoting to quizzes and video content instead.

I don’t expect clickbait to disappear anytime soon; it’s easy and provides a good return-on-investment, both of which are important to scrappy websites trying to market on social media. Or things could come full-circle and BuzzFeed could publish clickbait about making the best clickbait.

You can view and download all the BuzzFeed article data and metadata in this Google Sheet.

All graphics were generated using R. The charts were created using ggplot2 and the word clouds were created using the wordcloud package.

Could Markov chains be applied to any other data sets for hilarious effect? By using Slatton’s Python implementation of Markov chains plus 300,000 descriptions of public GitHub repositories retrieved from their API, I discovered that statistical randomness can indeed create funny innovation.

You can download a list of 1,000 Markov chain-generated projects here. Here are a few interesting ones:

• MaNGOS is a free, Open Source implementation of a tag at relatively random intervals.
• A Warhammer 40k simulator to teach myself both OpenGL and Clojure
• Perl interface to Git repositories via Ruby.
• A windows live messenger network client written in Erlang
• Rails plugin which allows to talk anonymously and use tripcodes if you want.
• A Firebug extension for displaying the latest from Hacker News
• Sinatra-inspired JavaScript node.js web development framework for lua. Inspired by rspec
• Inverted Index on top of Tornado
• Android LED interface library for various wave propagation techniques.
• CatchAPI is a Java API to remove the need for boring project setup.
• Adds basic social networking capabilities to your lighting system based on the concept of the Working with Rails
• Brute force your OpenERP data integration with flatfiles
• Culerity integrates Cucumber and Celerity in order to shutdown the computer.
• Parses ANSI color codes and converts them to iphone compatible mp4s using HandBrake
• A simple OFX (Open Financial Exchange) parser built on top of WordPress. Rolopress core theme

The code used to get the project descriptions from the GitHub API is available in this GitHub repository, and you can download the ~300k repo descriptions here. [5MB .zip]

Obviously, developers would prefer that their users not uninstall the software that they spent years creating. Regardless, developers should ensure that the uninstall process works properly.

Otherwise, a poorly coded PC game uninstaller could delete not just the game, but the entire hard drive. That would be a problem.

Ragnarok

In late 1998, Bungie (which would later create the Halo series) released Myth II: Soulblighter for Windows. Myth II, a real-time tactics game, was received with great reception and highly-positive reviews.

But before the worldwide release, a Bungie employee uninstalled version 1.0 of Myth II, and afterwards, her PC refused to reboot:

It was discovered by a very nice young lady in a Japanese office as she worked on the Asian versions of Myth – just as she was trying out the final build. She attempted to uninstall it from the main root folder of her hard drive – not brilliant, but pretty common practice – and perfectly fine for most games. The game proceeded to eat her hard drive like a fresh Bento Box.

The Myth II Uninstaller worked by deleting the folder which contained the game files. However, this implementation assumed that the user installed Myth II in the default directory. (i.e. C:\Program Files\Myth II). If the user installed Myth II in a different directory, such as the system root C:\, then when the user uninstalled the game, all the files in the C drive were deleted. Boom.

As Bungie later explained in a press release:

In certain unusual cases, using the Uninstaller can cause problems. If a user chooses an installation directory other than the default directory, they should NOT use the Uninstaller but should manually drag the Myth II folder into the Recycle bin and delete it.

Anyone who has the game but has not yet installed it is asked to wait for a new version of the Installer which we will release shortly. If you have already installed the game, there is no harm in playing it, but players should be careful not to use the Uninstall executable. If you have already Uninstalled and not experienced obvious problems, there is nothing to worry about.

Calling the potential obliteration of an operating system a “problem” is an understatement.

Although the issue was not widespread, Bungie decided to do a recall of all Myth II CDs and redistribute a version 1.1, eliminating most of the game’s profits in the process.

You could say that the Myth II version 1.0 uninstaller worked exactly as intended: it did indeed uninstall the game. It just worked too well.

UPDATE October 2014: A year after this article was posted, someone submitted this article to the Reddit subreddit todayilearned, where it subsequently went viral. In the comments, Reddit user rdmartell, a former developer at Bungie, posted more insight into the glitch:

So I wrote the Myth TFL installer (not this one). The reason this happened is because you could install other maps and whatnot, and to make the uninstall clean, we had to recursively delete your data folder. In the sequel, somehow that code (which did sanity checks) was used to delete the applications directory, instead of just the data directory in the applications directory. So if you installed it on C:/, it would delete everything recursively from there. Nowadays, uninstall usually leaves any user installed files (like saves or downloaded content) but at the time, as the article suggests, hard drive space was more precious. Also, I wasn’t there for this issue, so I don’t know the above for sure, but having written the previous installer, I’m pretty confident that’s what happened.

Later in the thread:

I actually think the bug was only there if you installed directly to root; not even in a folder. As I recall, that was the only way it deleted your entire HDD. Unless it was discovered later by going into the uninstalled code that it could happen in other cases (which I never heard about), I think the issue was a bit overblown by the media. Still, the last thing you want to be responsible for is nuking a customer’s hard drive.

Small world.

In Golden Sun, you play as a group of teenage mages called Adepts, who wield the power of the four elements (Earth, Fire, Wind, and Water) by casting spells with Psynergy. The antagonists of the game are a pair of jerks who are attempting to reawaken a dangerous and sealed power, and only four kids can stop them from destroying the world. Released two years later, Golden Sun: The Lost Age takes place immediately after the conclusion of the first game with a different quartet of Adepts and a new pair of jerks. In this sequel, it is revealed that the jerks who keep trying to kill you are not the bad guys and the whole conflict is just one big misunderstanding! Yes, the plot is very reminiscent of other Japanese-developed RPGs such as Final Fantasy and Chrono Trigger, but it’s still impressive for a game I typically played in the backseat of a car.

Like Final Fantasy and Chrono Trigger, Golden Sun has an inventory system where players earn new weapons and items by purchasing them at a store or by finding them as loot from defeating enemies in the world. The most powerful weapons, naturally, only drop from enemies at the end of the game with odds of less than 0.01% per kill. If you killed such enemies 100 times, for example, you would only have a 32% expected chance of seeing a rare drop among all the kills. And killing that many enemies could take hours. It depends on the almighty will of Golden Sun’s Random Number Generator.

As it turns out, the Golden Sun RNG is so poorly implemented that players have found out how to make events with 1/256 probability occur one hundred percent of the time.

You Make Your Own Luck

Soon after the first Golden Sun was released, players discovered that the first random enemy encounter the player enters after turning the GBA on was always a First Strike, where the player’s party gets a free turn. Was this a cute gameplay mechanic intended to give the player a quick advantage, or was it indicative of a flaw in the game’s RNG?

Most modern RNGs are partially seeded and randomized using a timer, i.e. the current time of the computer in microseconds. This makes it infeasable to reproduce the output of the RNG reliably.

However, the GBA does not have an internal clock, so time-based methods of seeding aren’t possible. By the time the sequel was released in 2003, the entire Golden Sun RNG was reverse-engineered:

Golden Sun uses the ANSI-C standard rand function as its Pseudo-Random Number Generator.

There is one difference: Golden Sun’s RNG uses an unsigned 4-byte number and returns it with no shifting, so you’d replace “return ((next »16) & 32757);” with “return (next & 4294967295);”

This, ordinarily, is a perfectly capable RNG, but there is one flaw with what Golden Sun does with it.

It only makes a new random number when something random happens. What’s more, it uses two copies of the RNG to create two random numbers, and the first RNG only handles battle situations.

As such, it is easy to control when a new random number is generated.

On top of this, the RNGs are seeded with 0 when the GBA is reset, and because of the way the game has been programmed, the RNG controlling battle events will continue to be 0 even when you load the game…

Golden Sun’s Random Number Generator isn’t “random” at all. It’s manipulated entirely from player-controlled actions, from the moment the Game Boy Advance is turned on.

Get Lucky

One of Golden Sun’s contributions to the typical RPG character class customization is the Djinn system. In the game lore, Djinn are impish creatures that grant enhanced powers to Adepts. There are four types of Djinn, one for each element, that can be found in the overworld, and each Djinni can be used to either increase a character’s statistics, or Unleashed to provide a powerful spell effect. Also, as with many other RPGs, Golden Sun implements an elemental rock-paper-scissors combat system: enemies are weak to an element and resistant to another.

A GameFAQs forum user by the name of “darkpanther” discovered that using a Djinn Unleash with an element that is the enemy’s weakness will not only increase the amount of Experience and Coins earned, but also quadruple the chance of the enemy dropping a rare item. (this fact is neither hinted anywhere in the game nor stated in any strategy guides)

This is essential toward manipulating the RNG. Normally, the rarest items such as Tisiphone Edge and Lachesis’ Rule have an “Item-Class Chance” of 9, meaning the item has a 1/(2^9-1) = 1/256 chance of dropping per enemy kill. The “DP Method” decreases the ICC by 2, meaning the new drop rate becomes 1/(2^7-1) = 1/64. But:

As ‘unpredictable’ as RNGs seem to be, if they are always seeded to the same value and do not use a timer-based response, they will become predictable by anyone who is good at noticing patterns. One pattern being, for example, that the 31st random number that is calculated - 33727075 - will drop any item if the enemy dying when this random number was called was just killed by an elemental Djinn attack that it is weak to.

Calculating the 31st random number would fail against an ICC9 item, but now works against an ICC7 item. (and all items of lower ICC too)

Therefore, in order to guarantee that an item drops, kill the appropriate enemy with an element-advantage Djinni such that the 30th random number is generated. (the 31st random number is generated on the enemy’s death)

How do you ensure that you only create 31 random numbers? From the Golden Sun Wikia:

Save the game and do a hard reset in the area where the monster that drops the item you need appears. Run around to get in a battle. If the encounter consists of a formation of 2 monsters, with at least one of them being the monster you want, proceed. If not, do another hard reset and try again.

During battle, have the entire party use multi-targetting Psynergy on the targets in the first turn, and since a battle right after a hard reset always involve your party attacking first where possible, the monsters will not be able to act. During the second turn, use two multi-targetting Psynergy on the enemies, and have the third party member (or fourth, if one of the three party members that act before uses Defend) unleash an attack Djinni of the element the monster is weak to on the monster. If by this point, the monster flashes in colour and dies, you will obtain the drop of that monster at the end of the battle, however rare it is. If the monster does not die then (either dies too soon or doesn’t die by this step), adjust the power of your attacks until you get it just right.

Moral of story: if you’re making a video game, seed your RNGs with an algorithm that players can’t manipulate 100%. Otherwise, hilarity will inevitably ensue. I forgive Camelot for implementing a simplistic RNG though; the two games’ awesome soundtrack more than makes up for it.

You play as Samus Aran, an armored bounty hunter who has infiltrated the base of the dreaded Space Pirates on Planet Zebes. Samus must defeat the leaders of the Space Pirates: the dinosaur Kraid, the dragon and memetic badass of the series Ridley, and the AI construct Mother Brain. Additionally, Samus must find a way to defeat the Metroids, which are nearly-indestructible parasites that threaten all life. Yes, this was a plot written in the 80’s.

Since the NES Metroid cartridge did not have the ability to save games, and the game itself took awhile to complete, Nintendo developed a password system to allow players to continue at a later time. Upon death, the player received a 24-character alphanumeric password that could be entered to restore all items the player has unlocked and remember which bosses the player has defeated. Most of these passwords were gibberish that required a pen-and-paper to write down and remember.

One password, however, is more memorable, and much more sinister.

The Password is Always Swordfish

Even before the days of the internet and personal computing, players found passwords with unique properties.

The Narpas Sword password:

is essentially the debug mode for the game, with all items unlocked, infinite health, and infinite health for Samus.

Another, more revealing password is the Justin Bailey password:

This password starts the game with most unlockables obtained, but more notably, it changes Samus’s sprite from an armored powersuit to a bikini swimsuit.

One password, discovered over 20 years later, long after the Metroid password system had been completely reverse-engineered, does not grant you all items. This password just crashes and resets the game:

WTF?

On a NES and various other rereleases, inputting that code will cause the game to either simply reset or transform the game into a glitched wonderland:

But what happens when you enter the password on an emulated Virtual Console, where the game environment is sandboxed? In the case of the 3DS Virtual Console, the 3DS completely crashes:

Rated E for Everyone?

You may be asking “did someone at Nintendo actually program a consequence for profanity in the game? Is the glitching and crashing a punishment for the perverse player?” In truth, the password is merely a coincidence.

As mentioned earlier, the password algorithm for Metroid has been reverse-engineered, and you can actually download a program to both generate and interpret such passwords here.

The profane password is indeed a legitimate password (in which Ridley is already defeated, ironically enough). But why does it crash the game? The Metroid Password Format Guide explains the secret:

There are five valid start locations. They are specified using bits 64-66. To start in Brinstar, all the bits should be off. Norfair, Kraid’s Lair, and Ridley’s Lair can be specified by turning their bit on and leaving the others off. Tourian is specified by turning on Norfair and Kraid’s Lair simultaneously. Any other combination of these three bits is invalid and will cause Metroid to reset.

The reset bit is part of the start location. There are 16 possible values, but only 5 valid ones. Turning on the reset bit will produce and invalid start location and cause Metroid to reset. This is why it is called it is called the reset bit, because it guarantees an invalid password.

For the profane password, the reset bit is turned on, so it will always cause a reset. And that’s why the weird behavior happens.

If you manually disable the reset bit using this tool in order to make the password valid, the new password becomes:

Not as catchy, unfortunately.

In fact, the Justin Bailey password is also a completely coincidental password. Sometimes, passwords can just do very unexpected things.