Nurse Carter – a COVID19 hackathon winner

So, when I built Nurse Carter years ago, I got the data from the Ministry of Health’s website in Trinidad and Tobago. I hadn’t updated the data since. Also, I built it with version 3 of the Microsoft Bot Framework.

Bot Framework updated a lot since then, and so did the health facility schedule. I took the opportunity to hit two birds with one code. I updated to the latest bot framework bits, changed from Node to dotnet and updated the health facility data.

To make it COVID-19 relevant, I included information about that as a menu option in Nurse Carter.

When I first did Nurse Carter, I feel like I had a lot more time – lol, I did because I was on “paternity leave” (it’s regular vacation I’m calling that). Now, in snatches of time, I found myself thinking “this is rell work!”

Nevertheless, I got to the submission line and Nurse Carter to a devpost hackathon and it was one of the top projects out of 1500 that were submitted! COVID-19 has brought a lot of heartache but I’m hoping that innovative approaches like Nurse Carter get more and more opportunities to deliver meaningful value in people’s lives.

That time when global quarantine liberated digital services. #CoronaEffect

There has been a sudden, dramatic step-wise improvement in digital services in Trinidad & Tobago.

• 

• 

• 

You can place an order via WhatsApp for groceries in Trinidad & Tobago now. From one of the largest chains in the country.

Many government agencies are available in a real-time way on social media. Providing updates of a COVID and even non-COVID nature

A local bank has started to enable sending of money that the recipient has access to in 1 day, down from 4 to 5. The others, I expect are sure to follow.

I’m here for all of it. Maybe this activity is just merchants and other service providers responding to the change in environment. Everyone’s online, and the only way to get people to interact meaningfully with your service is by meeting them where they are at.

For now, I’ll list these services as I hear about them. I won’t celebrate too loudly, because, you know, many people have been clamoring for this for a long time.

Parse a pdf, quickly

1. You have docker installed.
2. You don’t have time to futz around with PDFs

If you have a PDF you’d like to get tables out of, tabula works a dream. When last I used it successfully, I was on my mac. I jumped on to a windows machine, and I just couldn’t get it to run.

*a few googles later*

I found a docker command that hosts a web API that gives access to tabula:

docker run -p 8080:8080 -e HOST=0.0.0.0 gavinkflam/tabula-api:1.0.0

At the readme for this container, the example was this:

curl -X POST -H 'Content-Type: multipart/form-data' \
  -F 'file=@my-pdf-file.pdf' -F 'guess=true' -F 'pages=all' \
  http://localhost:8080/api/extract

Thankfully, because of WSL, I just popped over into bash and I had tables in a flash 😎

I’m a student, where can I Azure? (freeness welcome)

Whenever I teach cloud technologies, I emphasize practical applications, even for the newest students. This really helps the topic come alive and allows the students to engage the material in a more realistic way.

As it turns out, a few of my friends reached out to me about Azure resources for learners, so, I thought I’d just jot down a few resources here.

Azure for students

So, a simple search online should point you here:

https://azure.microsoft.com/en-us/free/students/

One of the first questions that come up is “Am I worthy eligible?”, which is helpfully answered in the FAQ:

Azure for faculty

Now, if you’re an educator, teaching assistant or other faculty, you can sign up for open source content – I’ve used this, it’s legit and the github repo is very active! As well as get a free Azure account for doing demos and showing how the stuff works.

Just plain old free Azure stuff

If you aren’t enrolled anywhere and just want to learn Azure stuff, there are also a lot of free-levels that you can use to get started.

https://azure.microsoft.com/en-us/free/

When you access this option, they’ll ask for a credit card number – this is for verification, they won’t charge you automatically unless you choose to upgrade. You have to choose to upgrade, it’s not going to sneak up on you.

Some azure services have levels that are always free, like Azure App Services, Azure Functions and Azure Kubernetes Service. There are others that are free for the first 12 months of usage. You can check out the full list here (scroll down).

So, there you go, get out there, get started, have fun!

Trinidad and Tobago Azure Users Group

We’re having the first of many (I hope) Microsoft Azure users meeting this Thursday (February 13).

How do you Azure?

I’m excited by this because here in Trinidad, I’m grateful that in and around Teleios, we’ve been using Azure for years, and I’ve often wondered about who else around us is learning, playing and building on the platform. Nigel Sammy, Christian Anton and I will be sharing on Azure in general, Azure DevOps and Azure Arc. It’ll be a really nice spread of features, from the novel to the staid.

So, convince you’re boss, if you’re interested, we’ll be at the Microsoft Office on Mucurapo Road, from 9:00 to 11:00 am.

The event is free, so’s the breakfast and you can register here.

Three things about January

Just before I went back out to work in January, I had a fair sense of how I wanted to start the year. At work, I’d be diving in to Planner and refining some early 2020 goals that Anand and I had come up with. At projects, I’d be doing some arduino and bot stuff. All great.

But from January 2nd, I found myself very deep in legacy code from at least a decade ago. And I was working very late. It’s like time stood still. When my team and I raised our heads for a week, a change to the environment around that same code meant even more speelunking.

Out of this last few weeks I saw three things of note:

try
{
//actual code to do something meaningful
}
catch
{
//empty, meaningless catch block
}

1. Stop using empty catch blocks.
   I haven’t had cause to leave dangly, useless catch blocks lying around for a few years, but I remembered when I used to. And boy was it a shocker. In the age of amazing tracing tools like App Insights, throwing away trace opportunities was like a slap in the face.
   When I showed a few developers at work some examples of empty catch blocks, they understood why it might have happened – I didn’t know what I wanted to do with the exception at the time – but my response was largely, “well, at least log it na”. At least. Even logging took a hit this month.
   
2. Log with sense
   The worst kinds of exception message in a log is “An exception occurred”. For the most part, sometimes, when looking for the flow of a problem, I might just put a log statement to show that we got to a place in code. Sometimes, that place is an error handler. The way I see exceptions now, if you’re logging that it occurred, take some time to log what that block was trying to do, as well as what parameters are available. In a messaging system for example, simply a message Id is a big deal. So, first we log, then we log with sense.
3. Sleep is very good.
   Not just sleep, but a firm commitment to not rush off with the first good idea you have when facing a problem. I have actually heard this stated another way, “Beware the danger of the first light”. While trying to debug our way out of the hole we were in, several ideas came up. Most recently, I had a great one that would have led to massive changes in the system. I was about to announce it, but one of the managers on our team cautioned me. She said it was Friday, sleep on it and see if you feel the same way on Monday.
   Monday came, I felt the same way. Until I spoke to one of the newest engineers on our team and realized we could achieve what I wanted to with a change to one module, as opposed to the whole system. That was good.
   Over the course of this month, there were a lot of challenges.

Making a living building mobile apps in Trinidad & Tobago and other stories

This blog post title sounds like a book. 😁.

Back in 20-some-teen, I built a windows phone app called Police Post. It started off as a reasonable idea – provide an offline version of the list of police stations in Trinidad and Tobago, with their locations and contact numbers.

I had some quirky app names back then… they eventually became quirky chatbot names, but more on that some other time.

I later jumped the shark by overlaying the map of police stations with information about murders that happened in the same region as a police station.

Even writing that makes me cringe a little. Back then, I was convinced, “This is a good great idea for the app”. Now, I’m like, “Why…..?”

I remembered Police Post while I was preparing to deliver a presentation at the Trinidad and Tobago Intellectual Property Office’s seminar on “How to make a living from mobile apps”. My focus was on the state of the mobile app development in TT.

I found while preparing for the talk that there was a lot to be said about how active trinbagonians are with their mobile devices, but not necessarily with local apps.

So, a stat like this would be familiar to a lot of people who care to do the research. There are lot of phones and a good bit of social media usage on those devices.

And, using the top free apps in the Google Play Store as a proxy, it only confirmed that we really like social (and Google Translate).

Rank	Name	Category
1	TTPS – Trinidad & Tobago Police Service	Social
2	WhatsApp Messenger	Communication
3	T&TEC Mobile	Communication
4	Free Phone Cleaner – Cache clean & Security	Tools
5	Messenger – Text and Video Chat for Free	Communication
6	Instagram	Social
7	Snapchat	Social
8	Facebook	Social
9	Wish – Shopping Made Fun	Shopping
10	D’Music	Music & Audio
11	TikTok – Make Your Day	Social
12	CallApp: Caller ID, Call Blocker & Call Recorder	Communication
13	King James Bible (KJV) – Free Bible Verses + Audio	Books & Reference
14	Google Play Games	Entertainment
15	Tubi – Free Movies & TV Shows	Entertainment
16	Netflix	Entertainment
17	Traffic Cam TT	Travel & Local
18	Facebook Lite	Social
19	Messenger Lite: Free Calls & Messages	Communication
20	Safe Cleaner Plus	Tools
21	Google Translate	Tools

Since I was concerned about making apps, as opposed to just using apps, I produced a list of the top apps by usage that were made by trinbagonians:

Rank	Name	Company	Category
1	TTPS – Trinidad & Tobago Police Service	TTPS	Social
3	T&TEC Mobile	Milsoft Utility Solutions	Communication
10	D’Music	Digicel_Group	Music & Audio
17	Traffic Cam TT	Trini Interactive	Travel & Local
41	TT RideShare	TT RideShare	Travel & Local
45	RBC Caribbean	RBC Financial (Caribbean) Limited	Finance
46	Pin.tt	Larixon Classifieds	Shopping
53	My Digicel	Digicel_Group	Tools
74	Scotiabank Caribbean – Banking	Scotiabank	Finance
76	bmobile Top-up	Powered by eTopUpOnline.com	Shopping
96	Caribbean Airlines	Caribbean Airlines Limited	Travel & Local
98	RepublicMobile	RepublicBankLimited	Finance

A more diverse list, pretty corporate, but seemingly high on the “getting things done” measure.

Both lists were a snapshot of top apps on November 10, 2019. The TTPS app was released the week before, and people were responding. TTPS seemed to have had a good push behind the app, so that’s good.

This may be why I remembered Police Post. Another reason that brought it into focus may have been because of these sentiments I got from Julie David, a Senior Policy Analyst at NIHERST.

Julie and her team have been working on a sectoral mapping of the software industry in TT, so I thought her insights might be useful.

They certainly were as they gave me a snapshot of the state of affairs that I recognized. Here are a few of those challenges:

1. Small companies
2. Lack of strong cohesion between business models & development
3. Lack of design & UX quality

When I built Police Post, it was a small app, meeting a specific need that made no assumptions about having a business case. So Julie’s feedback to me was on point. Around that time, one of my key goals was simply demonstrating capacity.

I was making the statement, yes, we can build apps, focused, useful ones. Now, I’m here to say, yes, we can build business on top of platforms that include mobile apps.

My presentation concluded with looking at stats on global Internet trends. Mary Meeker’s report on those trends was an excellent resource for this and I hope that we all would use it to inform our next steps.

The TweetWhisperer

Today is GDG DevFest 2019 in Trinidad. The organizers put out a call for sessions, and I was happy to share one of the ideas that had been rolling around in my head for a while.

I Facebook in pirate, don’t @ me.

So, here’s the TL;DR: my idea was to take my likes on @Twitter and funnel them into Google Keep. Along the way, I’ll automatically categorize the tweets and then confirm that categorization via a chatbot. Simple stuff.

So simple, I didn’t even use Visio to diagram it.

What I actually did:

Twitter Likes

I made an Azure Function that would periodically poll my twitter account and get the last tweets that I liked. To do this, I had to create a developer account in twitter to get the appropriate creds. The function was pretty simple:

	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Net.Http;
	using System.Text;
	using System.Threading.Tasks;
	using Microsoft.Azure.WebJobs;
	using Microsoft.Azure.WebJobs.Host;
	using Microsoft.Extensions.Configuration;
	using Microsoft.Extensions.Logging;
	using Newtonsoft.Json;
	namespace TweetWhisperer.Function
	{
	public static class TwitterFavoritesReader
	{
	static readonly HttpClient client = new HttpClient();
	[FunctionName("TwitterFavoritesReader")]
	public static async Task Run([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, ILogger log,
	ExecutionContext context)
	{
	var config = GetConfig(context);
	var token = config["TwitterAccessToken"] ?? await GetToken(config["TwitterKey"], config["TwitterSecret"]);
	var screenName = config["ScreenName"];
	//TODO: ONLY GET last minute of faves
	var uri = $"https://api.twitter.com/1.1/favorites/list.json?count=5&screen_name={screenName}";
	var authHeader = System.Convert.ToBase64String(
	new UTF8Encoding().GetBytes(token));
	try
	{
	client.DefaultRequestHeaders.Add("Authorization", $"Bearer {token}");
	}
	catch (Exception err)
	{
	log.LogError(err, "it happened");
	}
	var favoritesResponse = await client.GetAsync(uri);
	var favesJson = await favoritesResponse.Content.ReadAsStringAsync();
	var faves = JsonConvert.DeserializeObject<List<TwitterFave>>(favesJson);
	await CategorizeThem(config, faves, log);
	log.LogInformation(faves.Count()+" favorites found");
	}
	private static async Task CategorizeThem(IConfigurationRoot config, List<TwitterFave> faves, ILogger log)
	{
	var uri = config["CategorizeThemUrl"];
	var keepItResponse = await client.PostAsJsonAsync(uri, faves);
	var resp = await keepItResponse.Content.ReadAsStringAsync();
	log.LogTrace("Was it kept??? " + resp);
	}
	private static IConfigurationRoot GetConfig(ExecutionContext context)
	{
	var config = new ConfigurationBuilder()
	.SetBasePath(context.FunctionAppDirectory)
	.AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
	.AddEnvironmentVariables()
	.Build();
	return config;
	}
	private static async Task<string> GetToken(string key, string secret)
	{
	var uri = "https://api.twitter.com/oauth2/token";
	var bearerTokenCredentials = $"{key}:{secret}";
	var authHeader = "Basic "+System.Convert.ToBase64String(
	new UTF8Encoding().GetBytes(bearerTokenCredentials));
	client.DefaultRequestHeaders.Add("Authorization", authHeader);
	var nameValuePair = new KeyValuePair<string, string>("grant_type", "client_credentials");
	var encodedContent = new FormUrlEncodedContent(
	new List<KeyValuePair<string, string>>{nameValuePair});
	var result = await client.PostAsync(uri, encodedContent);
	var resultContent = await result.Content.ReadAsStringAsync();
	var e = JsonConvert.DeserializeObject<AccessTokenHolder>(resultContent);
	return e.access_token;
	}
	}
	public class AccessTokenHolder
	{
	public string token_type { get; set; }
	public string access_token { get; set; }
	}
	}

view raw TwitterFavoritesReader.cs hosted with ❤ by GitHub

Categorizing Likes

In the DotNetConf keynote a few weeks ago, I saw an ML.NET demo and I got the idea to use it here, too.

ML.Net to build models (easy peasy)

All my notes

I pulled all my notes in keep to train an ML model. It was very easy, particularly because I used gkeepapi, an unsupported library for interacting with keep.

Doing this made me glad that I could think in terms of a bunch of cooperating functions, because the function to extract the notes from keep was written in python, while most everything else is in C#.

	import logging
	import gkeepapi
	import jsonpickle
	import os
	import azure.functions as func
	class KeepNote(object):
	label = ""
	text = ""
	category = ""
	# The class "constructor" – It's actually an initializer
	def __init__(self, label, text, category):
	self.label = label
	self.text = text
	self.category = category
	def __json__(self):
	return {
	'label': self.label,
	'text': self.text,
	'category': self.category,
	'__python__': 'mymodule.submodule:KeepNote.from_json',
	}
	for_json = __json__ # supported by simplejson
	@classmethod
	def from_json(cls, json):
	obj = cls()
	obj.label = json['label']
	obj.text = json['text']
	obj.category = json['category']
	return obj
	def main(req: func.HttpRequest) -> func.HttpResponse:
	logging.info('Python HTTP trigger function processed a request.')
	keep = gkeepapi.Keep()
	clientId = os.environ["gclientid"]
	secret = os.environ["gkey"]
	# return func.HttpResponse(f"{username}")
	success = keep.login(clientId, secret)
	body = ""
	title = ""
	category = ""
	try:
	req_body = req.get_json()
	body = req_body["body"]
	title = req_body["title"]
	category = req_body["category"]
	except ValueError:
	pass
	else:
	body = req_body.get('body')
	title = req_body.get('title')
	category = req_body.get('category')
	print(body)
	print(title)
	print(category)
	#return func.HttpResponse(title)
	labels = keep.labels()
	currentLabel = None
	for label in labels:
	if label.name == category:
	currentLabel = label
	break
	if currentLabel is None:
	currentLabel = keep.createLabel(category)
	keep.sync()
	#return func.HttpResponse(currentLabel.id)
	if len(category) > 0:
	note = keep.createNote(title, body)
	note.color = gkeepapi.node.ColorValue.Pink
	note.labels.add(currentLabel)
	keep.sync()
	return func.HttpResponse(note.id)
	gnotes = keep.all()
	#for label in labels:
	#print label.id
	#help(label)
	keepNotes = []
	for note in gnotes:
	try:
	woi = "—>"
	data = note.title
	for label in note.labels.all():
	try:
	keepNote = KeepNote(note.title, note.text, label.name)
	keepNotes.append(keepNote)
	except:
	pass
	#print data
	#woi = woi + "," + label.name
	#print woi
	#print note.text
	#print "\r\n"
	except:
	pass
	#json = simplejson.dumps(keepNotes, for_json=True)
	json = jsonpickle.encode(keepNotes)
	logging.info(json)
	# name = req.params.get('name')
	# if not name:
	# try:
	# req_body = req.get_json()
	# except ValueError:
	# pass
	# else:
	# name = req_body.get('name')
	# if name:
	# return func.HttpResponse(f"{json.tostring()}")
	#http://www.convertcsv.com/json-to-csv.htm
	return func.HttpResponse(json)
	# else:
	# return func.HttpResponse(
	# "Please pass a name on the query string or in the request body",
	# status_code=400
	# )

view raw KeepIt.py hosted with ❤ by GitHub

KeepIt: A function to get my notes from Google Keep

The funny thing is, I didn’t really need the model. Most of the things I stored in keep, were in one or two categories – not very good for modelling. I guess I hadn’t really realized that. To be honest, I was probably storing things in keep that I had a very high priority on, which turned out to be mostly cloud things. Go figure.

How the bot will help change things

So, I’m grabbing my tweets, categorizing them based on history and preference and I’m ready to store them, except, as I’ve shown above, my categorization is whack. Thus, I also made a chatbot, that will take my liked tweets and ask me to adjust the category I’ve labelled it as.

TweetWhisperer: Helping me categorize my tweets better.

So, with these three components, a likes-harvester, a categorizer and a chatbot, maybe, I’ll get better at returning to these topics that I have an interest in.

An unexpected journey

LOL.

My family was going to the beach a few weekends ago, and so I decided to try and catch some miles on the way there. Literally, I’d jump out of the car just before we got there and jog the rest of the way.

We were going to Las Cuevas beach, which for some reason I thought was a short distance from Maracas beach. It’s not. It’s really not.

I mean, I try and do a long run on the weekend, so an extra mile on my usual 6 shouldn’t have been too arduous. However, Maracas Bay Village is the most hilly place I’ve ever run!

So, four miles in I was exhausted, but thankfully, I had my buds and my podcasts were on fire. Recently, I was interviewed on Tim Bourguignon’s Software Developer’s Journey podcast, so I subscribed to it and it turns out to be just what you need when you’re on a long run with no end in sight. Tim uses interviews to tease out compelling, interesting developer journeys. Which at times can feel just like my run felt. It was so good to have that distraction.

I was able to finish two episodes, one featuring Stephanie Hurlburt, which if I had to rename would be, The Importance of Balance. And one featuring Guillermo Rauch.

Just when I would have pulled a Geoffrey and jumped in a car to get the rest of way, I saw an old man who said Las Cuevas was one ridge over and it was!

Lightning in a Hansard bottle

Some of the technology team that brings the Hansard online in Trinidad & Tobago

When we built the Hansard Speaks chatbot in 2017, I was super excited and told all my friends about it. One of them now works in IT at the Parliament and he invited me to talk with the team about it.

At the brief talk, I spoke about the motivations for building the chatbot, how we thought it was a great way to win arguments about who said what in Parliament, and that we liked how easy it was to bring an automated conversational experience into that world.

I think the team at the Parliament does a great job. I’ve always liked that they were among the early movers to bringing Internet technology into governance. They’ve been online for a long time, they make copies of the Hansard available on their site and they stream proceedings. They’re also on Twitter and are pretty active.

We spoke about how much Hansard Speaks leverages cloud technology and the fact that though the government is progressing, the policy on public cloud means that they have to find ways to use on-prem tech to accomplish similar functionality. HS uses the Microsoft Bot Framework, Azure Media Services and Azure App Services. If they wanted to do that off the cloud, they could but it would just be a bit harder.

I’d love if they shared more about what they do, in terms of all the teams that go in to making the Hansard come alive. There’s a real production team around ensuring those documents get generated and that the parliament team can handle requests that come down from MPs about who said what, when.

Since it’s been two years after we first did the chatbot, I described to them one key change we might make if we were doing it again. We would use a tool like Video Indexer on the videos they create.

Video Indexer’s insights on a Parliament video from Trinidad and Tobago.

It would let us do more than simply get a transcript, as we would then be able to get information on who said what, and how much they contributed to a session. We would be able to get key topics that were discussed.

So, it was great to speak with some of the guys behind the Hansard and share with them ideas on services they can leverage to make their offering even more compelling, insightful and relevant.