A Software Engineering Course

A good conversation started because of that post on Caribbean Developers. (Also, my Facebook is in Pirate, don’t hate, lol).

When I saw this question, my first response was similar to quite a few others on the thread. “Teach a combination of these things: CI/CD; Source Control; DevOps; Observability and Telemetry; UX…”.

But I didn’t post it, because I had an inkling that something more was needed. That a three-month course to even masters-level students might not be sufficient to hit the mark. Then this hit me:

Which true Engineering discipline takes three months to understand?

I mean, Mendez’ question asked about a Software Engineering course. I get where he’s coming from. I did my undergrad at the University of the West Indies. And there was a Software Engineering course back then. It had a textbook and everything. That’s where I learnt about Waterfall, XP and UML. Things I barely used over the last 15 years. I have drawn a lot of boxes on whiteboards, though.

I think back then it was thought that to a bunch of “programmers” (really, it was a computer science programme), all that matters is knowing that software is “engineered” via “a process”.

But. If that university wants to play a role in shaping future software engineers, I think a course of that duration isn’t enough. Just like with other engineering disciplines, there’s a certain base of understanding a student might need before the concerns of modern software engineering is discussed, but then the diet of consideration needs to be relevant and balanced.

At Teleios, our software engineering involves using Scrum, Kanban, various forms of test automation, front-end, back-end and distributed system development. We are continuing to deepen our investment in DevOps, telemetry & observability and have CI/CD as firm goals. We have dedicated design teams who communicate in a variety of ways customer intents. We believe in continuously developing skill as engineers by training and experimentation. We use design patterns and we’ve made up our own.

All that to say, if your software engineering program doesn’t contemplate issues like that and above, what are you doing?

To go back to the original question, enough time should be given to properly understand the difference between git rebase and git merge. And why git’s so popular even. Students of software engineering should be given a sight of the whole and what part code plays in it all (hint: not everything). In terms of a software development process, whichever one is selected, you don’t really learn by simply reading the relevant manifesto. You really learn by doing.

You do Scrum. Or Kanban. Or whatever process. And it takes time for that doing to get meaningful. So, I think Mendez should return to the drawing board with fresh eyes on the goal and perhaps come up with a more valuable way to help his students understand the world of software engineering.

Going a bit further, when I first conceptualized this response, it took two paths. Either offer 2-4 different courses focused on elements of software engineering, so the whole MSc has a Software Engineering sub-focus, or, they embed SE concerns throughout the programme. With a deliberate goal of having students graduate with understanding what is involved in modern software engineering because almost every course had a component or a focus on some capability needed to effectively participate in software engineering.

Back to the Sky: Processing Satellite Data Using Cloud Computing

From time to time, I work with researchers on projects outside of the day to day, forms-over-databases stuff. Mind you, I like a nice form over a cool data repository as much as the next guy, but it’s definitely cool to stretch your arms and do something more.

So, when Dr. Ogundipe approached me about cloudifying her satellite data processing, I had to do a lot of research. She had a processing pipeline that featured ingesting satellite images, doing some data cleanup, then analyzing those results using machine learning. Everything ran on local machines in her lab, and she knew she would run into scaling problems.

Early on, I decided to use a containerized approach to some of the scripting that was performed. The python scripts were meant to run in Windows, but I had an easier go at the time getting Linux containers up and running, so I went with that. Once the container was in good order, I stored the image in the Azure Container Registry and then fired it up using an Azure Container Instance.

Like a good story, I had started in the middle – with the data processing. I didn’t know how I would actually get non-test data into the container. Eventually, I settled on using Azure Files. Dr. Ogundipe would upload the satellite images via a network drive mapped to storage in the cloud. Since I got to have some fun with the fluent SDK in Azure a while back, I used it to build an orchestrator of sorts.

Once the orchestrator had run, it would have fed the satellite images into the container. Output from the container was used to run models stored in Azure ML. Instead of detailing all the steps, this helpful diagram explains the process well:

Super simple.

No, not that diagram.

The various cloud resources used to process satellite data in Azure.

So, I shared some of this process at a seminar Dr. Ogundipe held to talk about the work she does, and how her company, Global Geo-Intelligence Solutions Ltd uses a pipeline like this to detect locust movement in Kenya or the impact of natural disasters and a host of other applications of the data available from satellite images.

status.[utility].co.tt

Recently, there was thunder and lightning and heavy rainfall in Trinidad. Trees fell, and there was a range of property damage.

In cases like this, the aftermath can involve service outages across a number of service providers. For me, it was power. There was a bright spark, a loud bang and then silence.

Right after the first jumble of thoughts, I remembered you can call the power company to find out about outages. In this case, it was 800-BULB. The operator was prompt, he helpfully indicated that it was “a big problem that will take a long time to fix”.

When we awoke hours later, there was still no power, but added to that, I could not reach the power company on the phone. Either a lot of ringing or busy signals. And it made me think, calling to find out about the status of service is an approach we should retire.

From a cloud perspective, we can readily find out if a given provider is up or down by checking one of their status pages. Both Azure and GitHub comes to mind.

If we had access to the numbers, I expect the power company’s lines didn’t stop ringing until the day after the event. How did they handle the increased load and calls? That could have been lessened if there was an automated way their customers could get answers. Both at the phone level, and on the web.

I expect that some of the more advanced utility companies here in Trinidad already have internal systems that can readily provide this information, it’s therefore a matter of going the last mile and making it accessible to the public.

Of course, if utilities use a standard mechanism for reporting, then what can then happen is the status of a number of public utilities can be known to the nation, without having to make any calls.

What a day that would be.

Build 2020: Top 5

So, Microsoft’s Build 2020 Conference was held a few weeks ago. And at Teleios, we normally try to check it out because you get a good sense of where they’re going with some of the technologies and platforms we use to build solutions.

This year’s effort didn’t fail to deliver, even though the format had to be drastically change because of the strict rules of social distancing most of the planet was under due to COVID-19.

On a run a few days ago, I heard Jeff Sandquist, CVP dev-rel at MSFT talking about what it took to put on the event, likening it to a production by a movie studio. It was a great interview on a podcast called Screaming in the Cloud, in which Sandquist shared his top 5 Build Announcements (starts at 41:40):

1. Microsoft Teams: See it as more than a communication channel; it’s a development platform.
   “Single largest development opportunity for developers across the planet”
2. Developer Productivity: A number of changes and new resources have been announced to make it easier, faster and more productive when developing solutions.
3. Power Apps: Continuing commitment to enabling citizen developers to build no-code solutions that frees up system and other software developers to focus on other concerns while the organization gets things done. There was a lot to see about Power Apps and Power Platform and build.
4. “Making the best damn developer box” – Scott Hanselman’s keynote highlighted many new improvements to windows itself and the development experience there that underscore Microsoft’s goals around improving the tools that support the development process. From the new Windows Terminal, to making 2020 (finally), the year of Linux on the Desktop.
5. Microsoft Learn. Learning how to get into Azure and other Microsoft development technologies can be a challenge, to put it lightly. But Microsoft has recognized this and deliberately made learning about it’s products a strategic asset. From easy on-ramps via videos on the new Microsoft learn site, to very low barrier-to-entry access to the cloud through Azure, Learn its where it’s at. More seasoned developers will be happy to hear too, that documentation is seen as a key part of the developer journey and this is reflected in the way they approach it.

It was helpful to hear Sandquist talk about Microsoft’s Founding Moment. That they started as a Developer First company and they’re staying true to their roots.

Caribbean Developers’ Salary Survey – Redux

I saw on the Caribbean Developers’ Facebook group someone looking to run a salary survey:

I dig Matthew’s approach, because instead of doing the survey and trying to see if there’s interest, he’s looking to see if there’s any interest first and then leaning into the work.

From the looks of it, interest is low, maybe it will grow, but I wouldn’t be surprised if it doesn’t.

In 2018, I had a similar idea, for the same Caribbean Devs group. I may have been inspired by the StackOverflow Developer Survey and my own curiosity as it relates to the job market.

Here’s my survey results from back then:

caribbean-developer-salary-survey-2017Download

Just about 40 submissions were made, which was instructive. I appreciated that if you do a salary survey in a small pool, it can be too revealing if you have access to each submission, even if the submissions are anonymized. If I were to do this again, I would bear that in mind, I might leave out a Company field, for example, in favor of Industry. I’d also not include a Job Title field and instead leave it at Category and perhaps Years in the industry. I might also include questions that get to the heart of project diversity, too.

So, this is mostly a cautionary tale about surveys like this. There are good points of guidance to get from economists and people in the social sciences to understand how to design things like this, beyond simply capturing the data.

Nurse Carter – a COVID19 hackathon winner

So, when I built Nurse Carter years ago, I got the data from the Ministry of Health’s website in Trinidad and Tobago. I hadn’t updated the data since. Also, I built it with version 3 of the Microsoft Bot Framework.

Bot Framework updated a lot since then, and so did the health facility schedule. I took the opportunity to hit two birds with one code. I updated to the latest bot framework bits, changed from Node to dotnet and updated the health facility data.

To make it COVID-19 relevant, I included information about that as a menu option in Nurse Carter.

When I first did Nurse Carter, I feel like I had a lot more time – lol, I did because I was on “paternity leave” (it’s regular vacation I’m calling that). Now, in snatches of time, I found myself thinking “this is rell work!”

Nevertheless, I got to the submission line and Nurse Carter to a devpost hackathon and it was one of the top projects out of 1500 that were submitted! COVID-19 has brought a lot of heartache but I’m hoping that innovative approaches like Nurse Carter get more and more opportunities to deliver meaningful value in people’s lives.

That time when global quarantine liberated digital services. #CoronaEffect

There has been a sudden, dramatic step-wise improvement in digital services in Trinidad & Tobago.

• 

• 

• 

You can place an order via WhatsApp for groceries in Trinidad & Tobago now. From one of the largest chains in the country.

Many government agencies are available in a real-time way on social media. Providing updates of a COVID and even non-COVID nature

A local bank has started to enable sending of money that the recipient has access to in 1 day, down from 4 to 5. The others, I expect are sure to follow.

I’m here for all of it. Maybe this activity is just merchants and other service providers responding to the change in environment. Everyone’s online, and the only way to get people to interact meaningfully with your service is by meeting them where they are at.

For now, I’ll list these services as I hear about them. I won’t celebrate too loudly, because, you know, many people have been clamoring for this for a long time.

Parse a pdf, quickly

1. You have docker installed.
2. You don’t have time to futz around with PDFs

If you have a PDF you’d like to get tables out of, tabula works a dream. When last I used it successfully, I was on my mac. I jumped on to a windows machine, and I just couldn’t get it to run.

*a few googles later*

I found a docker command that hosts a web API that gives access to tabula:

docker run -p 8080:8080 -e HOST=0.0.0.0 gavinkflam/tabula-api:1.0.0

At the readme for this container, the example was this:

curl -X POST -H 'Content-Type: multipart/form-data' \
  -F 'file=@my-pdf-file.pdf' -F 'guess=true' -F 'pages=all' \
  http://localhost:8080/api/extract

Thankfully, because of WSL, I just popped over into bash and I had tables in a flash 😎

I’m a student, where can I Azure? (freeness welcome)

Whenever I teach cloud technologies, I emphasize practical applications, even for the newest students. This really helps the topic come alive and allows the students to engage the material in a more realistic way.

As it turns out, a few of my friends reached out to me about Azure resources for learners, so, I thought I’d just jot down a few resources here.

Azure for students

So, a simple search online should point you here:

https://azure.microsoft.com/en-us/free/students/

One of the first questions that come up is “Am I worthy eligible?”, which is helpfully answered in the FAQ:

Azure for faculty

Now, if you’re an educator, teaching assistant or other faculty, you can sign up for open source content – I’ve used this, it’s legit and the github repo is very active! As well as get a free Azure account for doing demos and showing how the stuff works.

Just plain old free Azure stuff

If you aren’t enrolled anywhere and just want to learn Azure stuff, there are also a lot of free-levels that you can use to get started.

https://azure.microsoft.com/en-us/free/

When you access this option, they’ll ask for a credit card number – this is for verification, they won’t charge you automatically unless you choose to upgrade. You have to choose to upgrade, it’s not going to sneak up on you.

Some azure services have levels that are always free, like Azure App Services, Azure Functions and Azure Kubernetes Service. There are others that are free for the first 12 months of usage. You can check out the full list here (scroll down).

So, there you go, get out there, get started, have fun!

Trinidad and Tobago Azure Users Group

We’re having the first of many (I hope) Microsoft Azure users meeting this Thursday (February 13).

How do you Azure?

I’m excited by this because here in Trinidad, I’m grateful that in and around Teleios, we’ve been using Azure for years, and I’ve often wondered about who else around us is learning, playing and building on the platform. Nigel Sammy, Christian Anton and I will be sharing on Azure in general, Azure DevOps and Azure Arc. It’ll be a really nice spread of features, from the novel to the staid.

So, convince you’re boss, if you’re interested, we’ll be at the Microsoft Office on Mucurapo Road, from 9:00 to 11:00 am.

The event is free, so’s the breakfast and you can register here.