More FastMail programming woes

Published October 21, 2017 / by Andrew

The email on my Threadstr project has been broken for quite some time, but I’ve been too busy with other projects to get it working, since it’s not functional anyway. I finally got it working (but I haven’t gotten it pushed to the website yet), and I thought I’d share the fix here.

(Sidenote– When I did a search of “fastmail nodemailer” in Duckduckgo, my previous entry on the topic was the third result. That’s both kind of neat and kind of frustrating, because, apparently, nobody else was publically working on the problem.)

If you get an error message from Nodemailer that says “Sorry, the messagingengine.com server names are no longer available,” this is because FastMail made yet another breaking change. (I’m kind of getting to the point where I’m considering moving to another service if this keeps up.) Unfortunately, the most recent version of Nodemailer hasn’t been able to catch up yet. You could change ths source code, but that’s probably not a good idea, because it would be overwritten by the next npm update.

When creating the transport (i.e., when you use Nodemailer’s createTransport function), you can’t currently use the “service” : “fastmail” option, because it uses Fastmail’s old settings. Instead, you’ll want to enter the settings manually, which, for FastMail, is the following:

"domains":["fastmail.fm"],
"host":"smtp.fastmail.com",
"port":465,
"secure":true,
"auth":{"user":"user@example.com","pass":"somepassword"}

You’ll need to use your own username and password, of course.

I hope this helps anyone coming across this problem. It’s an easy fix, but it is just frustrating that it needs to be done in the first place.

A couple of headaches with CakePHP

Published June 21, 2017 / by Andrew

I thought I’d share a couple of headaches I had with CakePHP and the solutions that I found. I’ve tried running through this tutorial to get a jump on Cake, but quickly came up with a couple of problems while running it in Apache on Ubuntu 16.04.

If you’re new to running Apache on Linux, probably the first thing you’ll want to do is run these commands:

    sudo adduser $username www-data
    sudo chown -R www-data:www-data /var/www
    sudo chmod -R g=rwx /var/www

Replace $username with your own username. This will let you write to the /var/www/html directory without having to use superuser privileges, but you’ll need to log out and back in before the change takes effect.

That’s only related to CakePHP insofar as it involves Apache with PHP, though, and is something that you’ll want to do on any dev environment for PHP. These are the two headaches I came across:

bin/cake isn’t executable

If you get a “Permission denied” error when trying to start the terminal or run the bake command, this isn’t because you don’t have correct permissions (well, not quite). It’s because bin/cake isn’t an executable file. In my case, I’d been using Linux long enough to know that that was the problem, but anybody that’s more familiar with a Windows environment might be confused when they get to this step, because the tutorial doesn’t mention it.

It’s easily resolvable. While in the project directory, run this command: sudo chmod +x bin/cake.

URL rewriting is not working

This is a second one that’s not mentioned in the tutorial, and even the CakePHP’s article on this particular topic didn’t have the correct solution. This took me hours to track down, and the reason why is because there are actually two steps that need to be done, and no one webpage contained them both as part of a single solution.

This first step I got from a source that I don’t remember, unfortunately. Run this command in the terminal: sudo a2enmod rewrite

The second step is to do some file editing. I got this one from a DigitalOcean message board. You’ll need to open apache2.conf. If you’re using a GUI, you can open this up with any text editor you want as long as you have su privileges. If you’re on a terminal, I like to use Vim, so I open it with sudo vim /etc/apache2/apache2.conf. If you want something easier to use than Vim, I hear Nano is easy to use (though I’ve never used it myself), so this should open it: sudo nano /etc/apache2/apache2.conf.

You’ll then want to find an XML-ish looking block that looks like this:

<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

And you’ll want to change the AllowOverride None to AllowOverride All.

When that’s done, you’ll want to restart Apache with sudo service apache2 restart.

This was all that I needed to get CakePHP running smoothly on an Ubuntu Server 16.04 VM using Apache. I can’t really guarantee that it’ll work for you, but I can say that it worked for me.

Vim Overriding Settings Woes

Published May 9, 2017 / by Andrew

Probably every good program has a few irritating anti-features. One of Vim’s aggravating quirks is that it loads plugins after vimrc, which means that some of those plugins may or may not override some settings. (The most frustrating one for me is that I like to have formatoptions set to nothing, but many plugin devs have decided that they know my typing habits better than I do.)

Customizability is extremely important to me, and these plugins really begin to wreak havoc on my workflow.

Probably the best thing to do would be to track those plugins down and edit them manually. Whenever you find a setting that’s being overridden, in this example we’ll use formatoptions, run :verbose set formatoptions?. This will tell you what offending script set it.

Unfortunately, this will likely only be an option if you have root access, because the plugins are in /usr/share/vim/vimXX/* (where “XX” is the Vim version). I have this problem in Termux’s default Vim installation, but that’s resolvable because I have root access to edit these files. However, I’m currently in a situation with a client in which I don’t have root access.

There is a good workaround that I found online, but it’s not perfect. Unfortunately, the website has been down since before I even found the site, but the search results looked promising enough that I checked Google’s cache, and it revealed a pretty good solution. Kind of an obvious solution, but a good one nonetheless.

The direct link is here, but, as I write this, I haven’t been able to reach it (including while using a proxy server).

At any rate, since I don’t know how long Google keeps their cache up, I thought I’d go ahead and duplicate it here (with a few minor changes), in case the website is gone for good.

What we need to do is take advantage of Vim’s “after” functionality, so it will reload $HOME/.vimrc after every plugin is loaded. (You may have some luck with $MYVIMRC, but I’ve found that there are many cases in which $MYVIMRC isn’t even set.) There’s a simple BASH script to do this, and it’s so short that I usually just type it directly into the terminal instead of making an individual script file.

(TL;DR version: Start here)

Please note that there’s a good chance that you’ll need to change the directory path from /usr/share/vim/vim74 to something else. It was only recently that Vim updated to 8.0, and a lot of systems (mine included) don’t use it, and some are still on 7.1. You’ll just need to figure out what directory to use.

    mkdir -p ~/.vim/after/ftplugin
    rm -i ~/.vim/after/ftplugin/*
    for file in /usr/share/vim/vim74/ftplugin/*.vim; do
        ln -s ~/.vimrc-after ~/.vim/after/ftplugin/`basename $file`
    done

Now, Vim will automatically load whatever’s in $HOME/.vimrc-after after loading any plugin. I just put source $HOME/.vimrc in mine. This means it’ll resource my vimrc every time it loads a plugin.

This is not ideal because it means resourcing a settings file every time you open a new file, but, from a pragmatic standpoint, it helps a whole lot more than it hurts.

FastMail Programming Woes

Published April 23, 2017 / by Andrew

I went through a headache today when I discovered that emails were no longer being sent from Threadstr. I couldn’t figure out what had changed because I haven’t actually touched it in quite awhile. Both my local development and threadstr.com had stopped working and with the same error message: “Error: Invalid login: 535 5.7.0 Incorrect username or password.” I had changed no settings since the last time it was working on either, I had made no changes to the domain registration, and the username and password were correct.

Well, it turns out that something changed on FastMail’s side. I’m not exactly sure specifically what they did, but they no longer let you programmatically send mail using the same password that you use to log in to the frontpage. You now need to create a new password specifically for your application.

It’s actually a good idea, but I sure wish they had let me know about this before they implemented it, because it completely broke Threadstr functionality. I can’t even find an announcement for it– I had to dig through the settings in a wild madcap adventure to find something, anything that could have caused the service to suddenly stop working.

(Irritated as I am, I’m making no plans to change to another email provider, because they’ve still been great so far.)

At any rate, I thought I’d make a post in case this helps anybody else. If you can’t get FastMail to work with something like NodeMailer or PHPMailer, or if it suddenly stopped, it could be this. Below are screenshots of the FastMail settings that will solve the problem.

Use the password generated here instead of the regular password in your application, and you’re golden.

Some scripts to easily make a private VPN server.

Published April 4, 2017 / by Andrew

I spent my weekend learning about OpenVPN and how to set up a DigitalOcean droplet to act as a VPN server. This was inspired by recent congressional actions and the anger and frustration that followed from citizens. The short version is that you can use OpenVPN to hide your browsing history from your ISP, so they won’t have anything to sell. Additionally, though, you can use a VPN to protect your data while you’re using a public network to prevent man-in-the-middle attacks, like a coffee shop or a hotel network. Traveling professionals have been doing this since at least the 90s, and my goal, at the moment, is to make this accessible to pretty much any human being that uses a computer.

This is why I wrote two BASH scripts to automate the instructions given by DigitalOcean. These scripts should let you have a fully-functioning, private VPN server up and running in about twenty minutes. This is not you handing information over to a third-party VPN service that you don’t know if you can trust– This is your own VPN server, that you control, running on the highly-trusted DigitalOcean cloud.

I also created a YouTube video series to give an example of it being set up. As I say in the videos, I want a private VPN server to be as easy to make as a peanut butter sandwich.

If you’re interested in setting up a VPN, I hope this is helpful.

Patience, coffee, and naps

Published April 2, 2017 / by Andrew

There’s probably a morality tale somewhere in here.

A BASH script wasn’t working in the VM I was testing it on, and I thought sure the tutorial I was using was wrong. (I was getting a little bit irritated, in fact.)

I wrote out a question for Stack Exchange, but decided to sleep on it before posting it.

I looked at it the next day and I immediately saw what I did wrong. It was kind of obvious, actually. Stack Exchange would have been a waste of other people’s time.

I’m not exactly sure what the lesson here is, but it does remind me of something that I used to say when I was in grad school: The three most important things in math are patience, coffee, and naps. Patience because you’ll be able to understand something better if you take it slowly. Coffee because, well, coffee. Finally, naps because sometimes, as one of my teachers used to say, your brain can get full, and you just need to take a step back and take a break before coming at it with a fresh mind. I think this applies to programming as well as it does mathematics.

Ant v 1.1.0

Published March 29, 2017 / by Andrew

After some feedback from my sister (i.e., the end-user) as well as some changes that I wanted to make myself, I’ve updated Ant to version 1.1.0. Remember that the purpose of the program is to help you keep track of what you’re working on. That way, if you have billable hours, you’ll know what you were working on and when without having to guess “I think I spent about two hours on the Pensky file.”

Find both the install files and the documentation on the Github site: https://github.com/anorman728/ant.

It’s currently Windows-only because I wanted to have something in my portfolio that uses C# and I wasn’t able to either get Gtk# working correctly or get Mono to compile this project.

New features include:

Can populate Prompt Times based on an interval and subinterval lengths instead of one-at-a-time.
Can clear the Prompt Times list with one button.
If the output file is not readable (which will happen if you have it open in Excel), it will warn you.
Adding times is more keyboard-friendly.
Headers added instead of just starting the file with “Ant log file.”
Can add multiple messages instead of just one message.
Prompt is a little less intrusive (it pops up in the lower-right corner and remains on top, but can be ignored).

I’d also like to point out that I’m available for freelance/consulting work– If you would like a custom program, feel free to email me at anorman728@gmail.com.

Ant

Published March 27, 2017 / by Andrew

The reason I haven’t been too active in the past week or so is because I’ve been busy working on a Windows Forms project in C# called “Ant,” and version 1.0.0 is now available on GitHub.

This project is based on a VBScript project that I had at my last job. I needed to keep track of how much time I spent on each individual project, so I wrote a script to prompt me once an hour with “What are you working on right now?” It would then take that input and put it into a CSV file, so I could keep track of it in Excel. (It’s VBScript instead of PowerShell because I wrote it before I became familiar with PowerShell.)

I was describing this project to my sister, and she said it would be very useful for people with billable hours. In its incarnation at that time, though, it would be kind of weird to distribute, so I decided to make it into a project that could be put into production. I did it in C# just because my experience in C# is light, so this gives me a little more exposure to the language.

Initially, I was going to make this project cross-platform using Gtk#, but I wasn’t able to get Gtk# working correctly (it’s very buggy), so it’s now a strictly-Windows project using the traditional Windows Forms. This is kind of ironic since I primarily use Ubuntu.

Between installing Windows 10 in a VirtualBox VM (Virtual desktops! Finally!) and learning a few things about Windows Forms that I didn’t already know from VBA in Excel, I spent quite a bit of time on this project, and there’s still some more to come. You can see the upcoming features in the Ant.todo file.

Reco: Outlining

Published March 7, 2017 / by Andrew

I have a reco today that’s a little different. It’s not for a particular program, but for a methodology that I’ve been using for quite some time.

Beginning a new project can be daunting. Everyone already knows that. Trying to organize my thoughts is difficult, especially for larger projects. “How do I get XYZ accomplished?”

What I’ve been doing for years, and not just for coding, is simply making outlines of what needs to be done. My personal methodology has been evolving over all of the time that I’ve been doing this and has become pretty specific. The end result (as seen in the screenshot above) is a todo list that’s organized as an outline. I’ve actually developed a Vim syntax file that color-codes it, and that color-coding has become extremely useful. (This is the same format as my notes files on GitHub.)

The key is as follows:

> = Todo
+ = Done
* = Information
– = Canceled
: = Title
! = Important to remember
? = Unanswered question

This doesn’t follow the specific rules of outlines that are taught in English classes. For example, it’s not uncommon for me to have only one subpoint. There’s good reason not to do this in a slideshow presentation, but it doesn’t really apply to these outlines.

Despite how specific the structure is, the content is very fluid. Throughout the course of a project, what it starts out as in the beginning could be radically different from how it ends up as by the end of the project. It’s not a “This project must follow this exact course” outline, but, instead, is essentially my thoughts written down. Todo points may be added, information points will definitely be added, entire sections could be added or canceled (I try not to actually delete so I don’t accidentally lose important information), etc. This helps me to organize everything as well as just remember everything. Additionally, the indented format lets me use Vim’s indent code-folding method (or, in my case, an expr called “ModifiedIndent”, which you can see on my GitHub), and keeping these files helps me more easily pick up an old project months or years after I haven’t touched it if/when I need to.

Of course, I would never say “This is something you have to do.” That’s actually the case for all of my recos, but, in this case, it’s especially true. I’ve very much come to recognize Matt Hartley’s favorite catchphrase, “Different strokes for different folks.” This is something that came about for me while I was an undergrad in math– Start solving a problem by writing down everything I know. (Come to think of it, I may have picked it up from the way they do differential diagnoses in “House”.) It evolved from there and has become an essential part of how I think.

This last part is slightly off-topic, but the screenshot above is from a project that I’ve just started (hence all > and no +). It’s a program that I’m writing for my sister to help keep track of time, based on a VBScript program that I wrote for myself a while ago (at the time, I didn’t know PowerShell). I was describing how I had a script run once an hour and prompt me with “What are you working on right now?” Then it would store the result in a CSV file, which I could later open in Excel and count how many hours I spent working on any individual project. She said that would be very useful in her line of work, and, huzzah, a C# project was born. If it were just for me, I would probably just use either PowerShell with Windows Task Scheduler or BASH with cron, but, if I want to share it with others, I’d rather have one package do everything (and, in this case, a GUI program). I’m hoping to be able to use Gtk# to run it cross-platform.

That’s all for right now. Thanks for reading!

Threadstr update: User options page and some Node.js tricks

Published March 1, 2017 / by Andrew

I have a major update and some minor updates for Threadstr.

It’s still not meaningfully functional, but it’s far enough along that I don’t intend to wipe the database on updates anymore.

All functions in the Options page are now usable. (Or, at least, they’re supposed to be.) Additionally, there’s a favicon added, helmet is being used to secure the Express app, and forever-monitor is being used to reset the server when it goes down for whatever reason. Finally, I moved the MySQL command that simply keeps the database alive to another place because it’s possible that its previous location was causing a memory leak by opening up way too many connections and not closing them, and its frequency is now every ten minutes instead of every five seconds.

Next up is actually being able to create and add to threads– The Feature Presentation.