Tips and Tricks for CircleCI

I am not a developer, but I work with developer tools all day. This can be frustrating, because a lot of tools made for developers assuming knowledge that I’m lacking.

Of course, if that wasn’t the case there would be less need for technical writers and my work would be harder to come by!

In any case, After spending the last few days migrating my job’s documentation to use the CircleCI 2.0 configuration, here are some tips I picked up:

Non-interactive shells

This can be tricky if you don’t keep it in mind. Because CircleCI tests are run in a non-interactive environment, your output can get …funky. For example, here’s the output of sculpin generate, which we use to turn our markdown files into html, running locally through CircleCI CLI:

Here’s the output from the build on CircleCI’s platform:

Sculpin assumes an interactive shell in which it can rewrite its output as it goes. CircleCI’s environment is set for better logging, not better human readbility (as it should be).

This is of course a trivial example, as I could add --quiet to my generate command to make my CI output nicer. But that’s not always the case…

Rsync output on CircleCI

Part of my build process involves copying my compiled docs to a staging environment. Our scripts use rsync, the most common choice for transferring files from A to B.

The Problem

No matter what permutation of logging flags I gave rsync, the only output I could get from my CI builds was empty, or a list of every single file it was checking for changes:

Not a critical problem, but it certainly reduces the usefulness of the output.

The Solution

While I could pipe the rsync command directly through grep, that would interfere with the while loop I put rsync in, which checks for exit codes to determine success. Instead, I constructed my rsync command as follows:

rsync --checksum --delete-after -rtlzq --ipv4 --info=BACKUP,DEL --log-file=multidev-log.txt -e 'ssh -p 2222 -oStrictHostKeyChecking=no' output_prod/docs/ --temp-dir=../../tmp/ $normalize_branch.$STATIC_DOCS_UUID@appserver.$normalize_branch.$STATIC_DOCS_UUID.drush.in:files/docs/

That’s a lot to unpack, I know. The key flags are -q, which keeps the output quiet, and --log-file=./multidev-log.txt, to redirect output into a log file. Then, once my loop has exited, I can run:

cat ./multidev-log.txt | egrep '<|>|deleting' || true

My pipe to egrep searches for the characters < or >, or the word deleting. Using the OR operator || true means that even if the grep returns no results the line will still exit with code 0, keeping my script from failing.

Now my output is concise and useful!

(By the way, I learned a lot of useful information about rsync output from this blog post, run through Google Translate.)

Dockerize for background processes

IMHO, this tool is poorly named. While it’s purpose was originally for CI builds involving multiple docker containers (I assume), it’s helpful even in a single container environment.

The Problem

I was seeing intermittent build failures coming from my Behat tests:

Behat is configured to look at port 8000 for my Sculpin server to serve the files it tests. The Sculpin step wasn’t failing, but rather it wasn’t starting up fast enough.

The Solution

My initial response to this problem was to add sleep 5 to my Behat step, giving Sculpin more time to initialize. But then my friend Ricardo told me about Dockerize. Because my Docker image is built from a CircleCI image, it was already installed. So I added a new step in .circleci/config.yml:

- run:
    name: Start Sculpin
    command: /documentation/bin/sculpin server
    background: true
- run:
    name: Wait for Sculpin
    command: dockerize -wait tcp://localhost:8000 -timeout 1m
- run:
    name: Behat
    command: |
    /documentation/bin/behat

Dockerize checks for a service listening on the port, and retries if one isn’t found:

Now I have one less reason for my builds to fail…

 

Do you have any tips, tricks, or answers to common “Gotchas” for continuous integration testing? If so, please share them!

Make a site, NOW!

A month or two ago I registered eastcoastamphicar.com and set up a WordPress site on Pantheon. The eventual goal was to make a web presence for my friend Billy, who restores the historic amphibious cars, and gives rides to folks whenever he can.

Today he called me to let me know that he’d be filming tomorrow for CBS new out of New York. He said he’s doing it for the exposure, that they’d plug his email and phone number at the end of the segment.

Wait, what? His email is at yahoo.com, and his phone is a land line, no texts. Sounds like he needs that website, and he needs it now!

So I took some drone footage a friend captured, some photos I’d scanned from Billy, and made an MVP for eastcoastamphicar.com. And hopefully in another hour, once Namecheap renews their DNS records, I’ll have a nice professional email address for Billy to give CBS.

Oh, look at that. It’s past midnight. Off to bed!

Idle Hands are The Hipster’s Plaything

I’m a pretty modern person. I work remotely for a tech company, I’ve lost count of the number of computers in my home (laptops, old towers, Raspberry Pi’s, etc), I even have Tux tattooed on my arm.

And yet I find myself falling into the hipster trappings of my generation; I have a turntable, I vape, and I just bought a french press. And I wonder to myself, why am I doing this? I’m a scarf and v-neck shirt away from hipster bingo!
Continue reading Idle Hands are The Hipster’s Plaything

Organizing Photos

This post is on quickly sorting photo libraries. If you want to skip the backstory and jump right to the commands, click here.

I recently switched my personal file hosting/sharing to NextCloud,  from OwnCloud. Among other features and improvements, it will automatically sort uploaded photos from my phone into folders by date. 

This great new feature created a glaring contrast with the many years’ worth of previous photos, mostly all in a single directory, some sorted manually by event.  The process of grammatically identifying, sorting, and deduplicating was simple enough once all the tools were lined up, but finding them was not. What follows are the tools I used to get my photos in order.

Continue reading Organizing Photos

How to ask for help on IRC

Hi.
Hello?
Anyone there?
My server broke, can anyone help?

As a regular IRC user, I’ve seen this message many times. I’ve also seen the hateful responses these intrusions generate. IRC is an older communication form (relative to the Internet), mostly inhabited by those who’ve used it for decades, and are set in their ways.

IRC is also where you’ll find large communities of people all interested in specific topics, usually tech-related. If you’re an expert on MySQL, Ubuntu Linux, Python, etc, you’re most likely found in #TOPIC on at least one IRC network.

Which is why there are newcomers everyday, logging on to seek the wisdom of these software gurus. If you’re one of them, this post is for you; the IRC amateur who’s venturing to Freenode or OFTC for the first time to ask for some much-needed help.
Continue reading How to ask for help on IRC

Why Am I a Technical Writer?

Because of how many great open-source projects have empty readme files.

Because I like open-source software, but I’m not a developer. Even when projects are documented, it’s written for other developers. The assumed level of knowledge is that of the person who wrote it. I think like a user, not a dev.

Because when beginners enter issues on GitHub, they’re met with scorn instead of compassion. And when answers are given, the goal seems to be to give as little information as possible, and make the user chase down the real answer from clues and hints.

Because I hate being told “RTFM“. If I’m asking, I’ve already read the man page. The problem is, a man page generally only lists the options, and briefly explains what they are. Not how to use them, or why you would want to.

I’m a technical writer because I want to use great software without having to know how it was built in order to use it. Because the high learning curve associated with Linux comes in part because its users hoard their knowledge, and belittle those asking for a helping hand.

I’m a technical writer because I want everyone to be able to use the software they want.

Are you a technical writer? If so, why do you do it?

Where to learn online?

As mentioned in my last post, I’m taking advantage of this arbitrary point in the Earth’s rotation around Sol as a starting point for a renewed spring of educational self-improvement. Since then I’ve been looking into what online courses to take advantage of.
Below are some of my impressions, both old and new, of some of the online learning resources I’ve dabbled with.

Continue reading Where to learn online?

New Year, New Goals (just like everyone else)

My goal is to focus less of my free time on games, tv, and other wastes of time, and more on learning, reading, and strengthening my skills as a technical writer.

To that end I’ve been looking into several online coding courses. I’ve already tried some in the past, and never found one that suited me. But there’s always more options, and perhaps a permutation of two or more courses on the same subject might yield an education path I find more palatable.

The secondary advantage will be that I will have more notable things to blog about, bringing this blog back up from its slumber.

So wish me luck, and if you have any first-hand experience with self-learning on tech subjects online, please share it in the comments!

Childhood Effects on my Adult Sleep

Last night I went to Zappa Plays Zappa, the show where Dweezil Zappa and a group of very talented musicians performs Frank Zappa songs. It was an amazing show. Jen and I were especially impressed with Scheila Gonzalez, who played several instruments perfectly, including a downright amazing saxophone solo, and sang with amazing range.

Since it’s the 40th anniversary of the album One Size Fits All, they played most of the album before doing other hits. If you’re not familiar with Zappa or the album click here and listen to about a minute of it. We sat in  the third row left of center, right in front of the trumpet  / trombone / guitar player, so it was pretty freaking loud. Imagine the scene, and then guess what I did.

I fell asleep.

Several times in fact. I was following this loud, intricate cacophony of sound, and then suddenly I’m in a random dream sequence with the music in it, and then I open my eyes back at the show. I wasn’t tired before the show started, and after the album finished and they played more standard rock songs I was awake and alert again. Why would the most musically complex music I’ve ever heard performed live make me fall asleep sitting up?

Here’s my theory: as a child my parents would play lots of music, including a lot of Zappa. I would be put to bed at 7 or 8 and they would play One Size Fits All among others while I slept, in an old house with thin walls. Has my brain been trained to associate this music with sleep, to the point where it will put me under in a situation otherwise unconducive to sleep?

If anyone has any insight into this sort of phenomenon, please let me know!