Why Governments Block Startups

There are a lot of permits and regulations needed in order to open a hotel, in the same way there are regulations needed to open a taxi franchise. This is a major barrier to new entrants in these markets.

It's in the interests of the entrenched (hotel|taxi) franchisees to have this barrier to new entrants as high as possible, since they're already past it. So they lobby and push government to enforce and even make the barrier even higher for new entrants in order to preserve their hold over the market.

This, of course, stifles competition and decreases innovation. It gets ridiculous when even plumbers, manicurists, and barbers need certification and licensing.

It's always in the interests of entrenched players to increase regulation. Whenever you see regulations, certification and licensing required, it usually indicates a stagnant industry ripe for disruption.

Apprentice, Journeyman, Master

A bachelors' in computer science doesn't make you a good computer scientist or programmer any more than a bachelors' in physics makes you qualified to call yourself a physicist.

In the hard sciences you're expected to take your masters while assisting more experienced scientists in their research. That's where you learn the actual skills you need. Your bachelors' is just the background knowledge you need so you can start learning at the feet of scientists more experienced than you. Your PhD is where they hopefully let you loose to do some research or work on your own. That's the traditional apprentice-journeyman-master approach to education, and it's worked for centuries.

It's understandable that a lot of students that take up CS have to cut their education short by skipping the last two steps of the masters, doctorate process because the opportunity cost (4-6 years of wages) is just too great. Education ain't cheap, unless you're on a scholarship, and they might have families to feed. So they have to learn on the job, and hopefully find one with colleagues or a process that would give them a chance to learn the skills they might have gotten through formal education.

Rather than bemoan the inadequate experience or skills of a freshly minted CS bachelor's degree holder, it would be more productive for the more experienced developers to guide and train them, impress on them the value of continuous learning while at the same time showing compassion and patience as they get their first taste of real-world software development.

Nobody gets to be an master before first being an apprentice.

Progress bars with tqdm

At your shell:

$ pip install tqdm

Whenever you iterate over something, wrap it in a tqdm like so:

In [1]: from tqdm import tqdm

In [2]: from time import sleep

In [3]: for i in tqdm(range(1000)):
   ...:     sleep(0.01)
      100%|█████████████████████████████████████████████████████████████████████████████| 1000/1000 [00:10<00:00, 98.49it/s]

In [4]:

Programmers don't understand Economics

On the phackers slack #careeradvice channel, a recurring theme seems to be the higher salaries that developers overseas are paid.

It always starts out with a developer -- usually a novice but not always -- discovering that developers in some other country with the same skillset as him are getting paid 5x (or whatever multiple) he is getting here. Other developers will then helpfully remind him of the greater cost-of-living, higher taxes, etc. in those countries. Invariably someone will say that Pinoy developers are worth more than what they're getting now.

Well, here's an important concept that I have had to explain multiple times:

You're worth whatever the market determines you're worth.

People often forget or ignore the fact that it's not just them and the employer that are negotiating. It's other employers, AND other workers. If you're negotiating with company, do you honestly think you're the only candidate they're considering? Other candidates or applicants may have the same skillset as you, but offer to work for a lower salary (for whatever reason). Other workers competing for the same jobs determine what you'll end up getting as well.

The real way to command a higher rate, regardless of whether it's local employment or remote, is to be as @mparaz and @radeinla put it, the special snowflake. If you're a commodity programmer, you get the commodity programmer rate because there are so many other candidates with the same or similar skillset.

He's been out of the job market for a while.

If you have, or can signal that you have a particular set of skills and there is a demand for those skills, then congrats, special snowflake, you can command a higher rate. The more difficult to obtain those skills (or signals) are, and the greater the demand, the better able you will be to command a higher rate whether here or abroad.

Google IO Extended 2017

For developers unable to make it to the annual Google I/O conference, Google holds I/O Extended events in cities around the world. This year Google Developer Group Philippines went all out and held I/O Extended events in five major cities, demonstrating their new APIs and services available to developers.

For the Manila leg, I ran a workshop for Google Cloud Platform's Natural Language API, for an audience of mostly students. In retrospect, NLP is too advanced and specialized field for undergrads or novice developers, unless they have a specific application or are using NLP already.

A number of the students couldn't get past signing up with GCP, which requires a credit card.

In the future, I'll need to be more upfront with the prerequisites and level of experience needed to get the most out of my workshops.

AI and Machine Learning for Software Engineers at Dev Festival

Yesterday morning at Microsoft Philippines, I gave a short talk introducing AI and machine learning to software engineers. Instead of walking through my usual R or Python code, I did a demo using Microsoft's own ML Studio, which was helpful for visualizing the machine learning process.

Most of the Q&A was about the possible impact of AI on jobs, inequality and security/privacy.

Installing kerasR

This is a quick reference to installing kerasR, a slim wrapper around Keras starting with the required Python packages.

Python packages

Create a virtualenv:

$ virtualenv pydata --python=/usr/bin/python3
Running virtualenv with interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/brian/pydata/bin/python3
Also creating executable in /home/brian/pydata/bin/python
Installing setuptools, pip, wheel...done.
$ source pydata/bin/activate
(pydata) $

Install keras. This will also install the other prerequisites for doing any sort of datasciency stuff in Python (numpy, pandas) as well as Theano. Tensorflow will be installed in the next step.

(pydata) $ pip install keras
Collecting keras
Collecting six (from keras)
Using cached six-1.10.0-py2.py3-none-any.whl
Collecting theano (from keras)
Collecting pyyaml (from keras)
Collecting scipy>=0.14 (from theano->keras)
  Downloading scipy-0.19.0-cp35-cp35m-manylinux1_x86_64.whl (47.9MB)
    100% |████████████████████████████████| 47.9MB 27kB/s
Collecting numpy>=1.9.1 (from theano->keras)
  Downloading numpy-1.13.0-cp35-cp35m-manylinux1_x86_64.whl (16.9MB)
    100% |████████████████████████████████| 16.9MB 66kB/s
Installing collected packages: six, numpy, scipy, theano, pyyaml, keras
Successfully installed keras-2.0.4 numpy-1.13.0 pyyaml-3.12 scipy-0.19.0 six-1.10.0 theano-0.9.0

Install Tensorflow:

(pydata) $ pip install tensorflow


In R, install the kerasR package:

> install.packages("kerasR")
Installing package into ‘/home/brian/R/x86_64-pc-linux-gnu-library/3.4’
** testing if installed package can be loaded
successfully loaded keras
* DONE (kerasR)

This may also install the reticulate package, which is an interface to Python objects and methods.

A guide to using kerasR is provided as a vignette.


If you get an error message when executing library(kerasR) saying:

> library(kerasR)

keras not available
See reticulate::use_python() to set python path,
then use kerasR::keras_init() to retry

this means kerasR (or more specifically, reticulate) can't find the keras python package, you need to start R after loading your virtualenv:

$ source pydata/bin/activate
(pydata) $ R
> library(kerasR)
Using TensorFlow backend.
successfully loaded keras