Sorting IP Addresses With PowerShell

I was recently attempting to sort a list of IP addresses that I had in a text file. PowerShell is awesome at sorting, so I figured I would use it given that I had nearly 2000 of them. I initially just tried to pipe the contents of the file to the Sort-Object cmdlet:

Get-Content -Path .\ipListFixed.txt | Sort-Object

The results were, suffice to say, lackluster. I ended up with something like this:

10.1.69.1
10.1.69.12
10.1.69.148
10.1.69.149
10.1.69.17

Gross, right? Obviously 10.1.69.17 shouldn’t be after 10.1.69.148 or .149. The issue is that the whole octet isn’t being considered. It’s being sorted like they’re just strings rather than IP addresses; 7 is bigger than 4 so they’re sorted accordingly. PowerShell isn’t really comparing the numbers of 17 to 148, for example, to realize that 17 is actually smaller.

It makes sense; PowerShell can’t assume that the value is an IP address so it’s treating the value like a string instead. This means that the key is to tell PowerShell it’s an IP address so that PowerShell can adjust the way it does sorting. This is possible by casting the values as the .NET Version class. Since I happened to be pulling the IPs from a file, I did the casting within the -Property parameter that I added to the Sort-Object cmdlet:

Get-Content -Path .\ipListFixed.txt | Sort-Object -Property { [System.Version]$_ }

This yields significantly better results:

10.1.69.1
10.1.69.2
10.1.69.12
10.1.69.17
10.1.69.29

Stay pink!

Unusually Pink Updates: Dropbox, Password Managers, and Ad Blocking

This will be one of those weird amalgam posts featuring multiple only mildly-related topics but which are too short to be a post in their own right. I though it would be helpful to provide some updates on a few things that Brandi and I have talked about in some of our past episodes. They’re all tech-related… but you already know that from reading the title.

Dropbox Upgrades and Price Increases

On June 1st, Dropbox added more features to their Plus plan. I’ve been a Plus customer for a couple of years now. It’s less of a “I need to pay for the extra space” situation and more of a “I store some important shit in here and this gives me more peace of mind” situation. The main features from the upgrade, according to the email they sent me, are:

Double your storage—save everything with 2 TB (2,000 GB).
World-class sync technology—move out-of-date files off your computer’s hard drive and to the cloud with Dropbox Smart Sync.
Dropbox Rewind—roll back accidental changes to any folder, or your entire account, up to 30 days.

The extra storage is nice, though suffice to say it’s not exactly something I need.

dropbox_space.png

Rewind seems really nice, though. I store a lot of PowerShell scripts in Dropbox, so I could see rewind being useful if I accidentally break one or need to snag an older version for some reason. While I’ve recently been making an effort to check in my bigger, more important scripts with Azure DevOps for version control, smaller ones that I’m the only person likely to use still just go into Dropbox.

Things aren’t all roses and unicorns, though, as the added storage and features come at a cost: $2 USD more per month. This bumps the monthly price up to $12 USD. The good part is that if you go with yearly pricing, you save that $2 a month and pay $120 for the year. I’m actually still on monthly pricing despite having used the Plus service for a few years so I’ll need to move over for sure. While the bonus storage is available now the new pricing isn’t happening for me until July. So before early July I need to swap to yearly billing to avoid paying the extra couple of bucks.

Wired Password Manager Recommendations

In our last episode, Brandi and I spoke a bit about password managers. Right after we recorded that episode Wired published an article all about password managers. The timing seemed perfect to share it. Of the ones Wired recommends Dashlane is the only that I haven’t used. It also makes me feel good that my preferred password manager, 1Password, was Wired’s #1 recommendation.

Speaking of Wired, I love their content but always run into my 5 free articles per month limit. I just happened to see earlier today that they’re doing a deal where you can snag a year’s subscription for just $10. This includes the print editions plus digital content (normally $50 a year) or just digital if you don’t care about print (normally $30 a year.) I was fortunate enough to see it right when they posted to Twitter. I have no clue how long this is running for, but it seems pretty worthwhile if you’re a geek and like tech news. And no… we don’t get anything for promoting this. I just like Wired.

Browser Ad Blocking

Also from our last episode, we discussed how Google was initially thinking about making changes to the webRequest API that would essentially cripple ad blockers. Like we discussed in the episode, after those initial reports and backlash in January Google had backed off a little and said they would think through how this API update should work. At the end of May, though, it came to light that Google still plans to nuke the webRequest API pieces that allow current ad blockers to work… that is unless you’re a G Suite customer.

What’s worse is that these proposed changes seem to be in the open-source Chromium project. So unless Microsoft does some work on their fork, the Chromium-based version of Edge could also be impacted. Suffice to say this gives me some significant pause on my previously mentioned plans to buy a new Chromebook. I’m now re-evaluating what I should do for my laptop situation. And if you haven’t used Firefox Quantum recently, now might be a great time to check it out.

PowerShell: Export Non-Standard and Multi-Value AD Attributes to CSV

I’ve recently been spending a lot of my time at work doing PowerShell scripting for a project that’s currently underway. While working on a bit of code late last week I ran into two interesting problems that I had yet to really stumble across despite doing heavy PowerShell scripting against Active Directory for nearly a decade now.

Exporting Non-standard Attributes To CSV

My first issue was that I needed to export some non-standard attributes from contact objects in AD to a .csv file. When I say “non-standard” I don’t mean that we’ve done a custom schema extension or anything like that. Instead, I simply mean that the AD PowerShell module only wants to include certain attributes when exporting AD objects to a .csv file. For example, if I run Get-ADObject -Properties * against a particular contact object and simply let the output dump to my shell, I see all of the attributes I expect including stuff like givenName and sn. However, if I run the exact same cmdlet but pipe the result to Export-Csv rather than sending it to the shell I will not have all the same attributes; ones like givenName and sn will now be missing. That’s a problem.

Storing Mutli-Value Attributes In a CSV

The other problem was that one of the attributes I needed is proxyAddresses. This is a multi-value attribute as contacts could very likely have multiple proxy addresses associated with a particular mailbox. They may have an alias, x500 addresses, etc. If you use PowerShell to pull this information from AD and dump it to the screen you’ll see the expected proxy address information. If you call GetType() on the proxyAddresses attribute you’ll see it listed as an array. That’s why if you export a contact to a .csv file and include proxyAddresses, the value for each contact in that particular column will simply read:

Microsoft.ActiveDirectory.Management.ADPropertyValueCollection

PowerShell isn’t sure what to do with a multi-value attribute as far as the .csv file goes so it stores the data type. This is also a problem!

The Solution

My solution to this was the following. Note that it’ll be easier to view the Gist for it.

# Get the objects with the required attributes and storing proxyAddresses properly in the .csv file.
Get-ADObject -Filter { objectClass -eq "contact" } -Server myserver.mydomain.net -SearchBase "OU=Contacts,DC=mydomain,DC=net" -SearchScope OneLevel -Properties name,givenName,sn,displayName,telephoneNumber,proxyAddresses,targetAddress,mail,mailNickname,company,department,l,physicalDeliveryOfficeName,postalCode,st,streetAddress,title | Select-Object name,givenName,sn,displayName,telephoneNumber,targetAddress,mail,mailNickname,company,department,l,physicalDeliveryOfficeName,postalCode,st,streetAddress,title,@{name="proxyAddresses"; expression={$_.proxyAddresses -join ";"}} | Export-Csv -Path .\sample.csv -Encoding ASCII -Append -NoClobber -NoTypeInformation

# Shows how to then use the proxyAddresses attribute in the future.
$allContacts = Import-Csv -Path .\sample.csv
foreach($singleContact in $allContacts) {
    $proxyAddresses = $singleContact.proxyAddresses.Split(";")
}

The first thing I needed was to get the properties I actually needed as opposed to just the ones PowerShell felt like including due to the object type. The solution was to:

  1. Specify all of the attributes I wanted in -Properties to ensure they would be available.

  2. Pipe the AD object to Select-Object and then re-specify each of the properties I wanted.

This works because PowerShell is selecting the attributes to include in the .csv file based on the type of object. Select-Object takes whatever comes through the pipeline, though, and makes a brand new, generic object from it. That object will include any properties from that original object that you specify, hence why I tell it the exact listing of properties that I need again.

This leads to the solution to the second problem which is what to do for proxyAddresses. PowerShell allows for the use of an expression to parse together a new property. In this instance the expression is to take the existing proxyAddresses array and -join it into a string. Each of the items in the array are separated by a semicolon. If a contact has the following proxy addresses, for example:

SMTP:pretty@unusually.pink
smtp:awesome@unusually.pink

Then the property in the .csv file will be stored as:

SMTP:pretty@unusually.pink;smtp:awesome@unusually.pink

You could pick any character for the delimiter that you want as long as it isn’t used in other data that you’re storing.

In this instance my goal was to take the exported data, re-import it, and then create contacts in a different AD forest. When creating a new object AD is expecting an array for the proxyAddresses attribute. A string with a random delimiter won’t do it. That’s where the second part of the example comes into play. If I’m looping through all of the data stored in the .csv file I can recreate the array of proxy addresses by splitting the string on my delimiter (in this case a semicolon) and assigning the result to a variable. That variable will be an array which can then be used to popular proxyAddresses when running New-ADObject

BlizzCon and the AXS of Evil

As we mention in our latest episode, our plans have changed and we will not be going to BlizzCon this year. We could just let it go and accept that it is what it is… but remember how I mentioned my love of throwing salt into the Internet? So we’re gonna do that instead!

AXS: The most useless ticketing service

We mentioned a few of our issues with AXS in Episode 5. It’s the new ticketing service that Blizzard decided to go with this year. Using AXS was a first for them; last year they went with Universe. In theory, it seemed like a good idea for a convention as popular as BlizzCon where there’s always a massive scramble to get tickets. Instead of having to hope you’re lucky enough to click on the button to buy tickets within 0.23 seconds of it going live, AXS is supposed to serve as an equalizer. They open up a lobby 30 minutes before tickets are available. Anyone who is in that lobby at any time before tickets go live is supposed to have an equal chance at getting them. Hopeful customers are randomly selected from the lobby and are given 7 minutes to purchase their tickets. Seven minutes is plenty of time, but if you don’t purchase them within that allotment then you’ve lost your opportunity and someone else from the lobby is given a chance at them.

The problem is that it didn’t fucking work at all. Instead of getting tickets when I submitted the purchase after getting lucky RNG, I got this error. Over and over and over.

axs_is_trash.png

I immediately opened support cases with both Blizzard and AXS. Blizzard was insanely unhelpful. They first asserted it must’ve been an issue with my card. I told them I tried 3 different cards with the same results, and that I saw pending transactions for each of them. They then told me it would have to be handled through AXS. This isn’t surprising, but I was hoping they would be a bit more accommodating considering it was an error on the side of the partner they were ignorant enough to select. So I then waited to hear something on my support case with AXS. And waited. And waited. And waited.

Days went by without hearing anything, so I hit them up on Twitter where they asked me to DM them. I sent them my case number and they responded with the following lovely exchange:

axs_dms_edit.png

Essentially they just admitted that their service didn’t do the one thing it’s supposed to do. On top of that, they didn’t even bother to read my reply, let alone respond to it. As I point out with the arrows I drew in the screenshot above, the blue check next to my original message means that it was “Seen”. The grey check next to my follow-up questions simply means that it was “Sent”. If that isn’t garbage-tier customer service, I don’t know what is.

Also, this apparently counted as covering the support case I opened as I never received an email or anything else in regard to it.

Despite this setback, we were still planning to go. Brandi had been in touch with someone we met at BlizzCon last year who has a friend who works for Blizzard and thus was supposedly able to hit us up with tickets. They weren’t on lockdown yet since internal folks apparently get their tickets later on, but we figured worst-case scenario we’d get to hang out in southern California for a few days. It was a nice bonus because I we got a great deal on a hotel… until we didn’t.

Hilton us where it hurts

If you’ve ever been to BlizzCon before, then you know the Anaheim Hilton is the place to be. Along with being right next to the Anaheim Convention Center for easy access to BlizzCon, it’s also the after-hours social hub of the convention with parties breaking out each night. The Hilton embraces this and last year had a bunch of pop-up bars in the lobby selling Bottle Logic’s special StarCraft 20th Anniversary beer. It was awesome.

sc_beer.jpg

Last year we didn’t stay at the Hilton because, on top of just being a Hilton, event prices are pretty insane. Staying for just 4 days, for example, easily pushes over $2,000 USD. I have family who works for Hilton which means I can often get a “friends and family” discount, but events are always blacked out. Brandi and I just happened to check this year, though, and sure enough discounted rooms were available. Suffice to say we pounced on one, booking 5 days for half the price most people were paying for 4 days. Even if something fell through without our BlizzCon tickets, being in the area and in the epicenter for the happenings of the convention still seemed super fun. Unfortunately, a week later I got the following email:

We are contacting you regarding your upcoming Go Hilton reservation at Hilton Anaheim - Confirmation # XXXXXXXXXX. Unfortunately, due to a misconfiguration in our booking system there were Go Hilton rates offered that should have not been made available. This affected reservations booked between May 09th and May 13th 2019. We regret to inform you that we can no longer honor these reservations, and they will be cancelled shortly. We are deeply sorry for this inconvenience and appreciate your understanding due to the systems error.

Followed shortly by a cancellation email with the most tone-deaf greeting possible:

JOHN, we’re sorry to see you go! We hope you allow Hilton Hotels & Resorts the opportunity to serve you in the future.

When I let Brandi know, we were both pretty fed up with the whole ordeal. Along with losing the insanely good Hilton pricing, now a week had passed since most people would have booked their rooms… meaning that the availability of something else in the area had already been shrinking. We would have to settle for a worse room at a higher price. Did we really want to spend the money and go through the hassle when neither of us having even played a Blizzard game in the past three months? Ultimately the answer was no, so we’ll be bailing on the convention this year.

With the money we aren’t spending on BlizzCon, Brandi and I will likely do other cool shit this year. I have a few other trips in mind that I’d like to take, and Brandi is getting a freaking puppy! Expect TONS of puppy pics on Instagram when that happens!

In the meantime, keep the Morton’s flowing and stay pink!

It's All About Open Source

A friend of mine who has been doing some programming (hi, Fy!) recently asked me for some ideas on open source projects they could look at and possibly contribute to. It made me think of what open source software I tend to use on a regular basis… which is a decent amount as someone who runs a few different Linux systems. It seemed like a solid opportunity to compile the Super Official List of Open Source Stuff John Likes™. As someone who doesn’t really know much about writing code, I have no idea if any of these are good candidates for someone to start contributing to open source; they just happen to be things I use on the regular. Without further adieu and in no particular order:

Notepad++

Notepad++ has been one of my favorite Windows text editors for as long as I can remember. While it may not be as flashy or as popular as some of the other editors I’ll get to in a little bit (yeah… I’ve got a thing for text editors), it’s hard to top the speed and footprint of Notepad++. It also has at least basic syntax highlighting for almost every programming language under the sun. I also can’t overstate the importance of being able to open a plaintext file at work and having the editor launch immediately.

Visual Studio Code

VSCode is another rare example of Microsoft software I enjoy. While I dislike the large, clunky IDE that is Visual Studio, VSCode is a nice balance between a simple editor and an IDE. It has a large plugin ecosystem that can give you benefits of things like Intellisense and a debugger without being needless. It’s based on the same Electron framework as GitHub’s Atom editor. I’ve found it to be such a nice editor across a wide array of languages that I’ve even taken to installing it on Linux systems for the times when I want a GUI editor.

If you’re curious how I configure it, here’s my settings.json file. Note that I mainly use it for PowerShell:

{
    "workbench.colorTheme": "Dracula",
    "editor.minimap.enabled": true,
    "editor.minimap.showSlider": "always",
    "editor.wordWrap": "on",
    "editor.scrollBeyondLastLine": false,
    "files.hotExit": "off",
    "powershell.integratedConsole.showOnStartup": false,
    "editor.fontFamily": "'Fira Code', 'Fira Mono', monospace",
    "editor.fontLigatures": true,
    "git.ignoreMissingGitWarning": true
}

Vim

Of course, most of the time when I’m using Linux I’m not using a GUI. I feel at home on the command line, which is good considering I run a handful of headless servers that I only ever access via SSH (which happens to be a great way to mess with scripting or code from a Chromebook if it doesn’t have support for Linux apps yet.) Vim is the de facto CLI editor for me (sorry Emacs users), and I can install it on literally everything. Most Linux distros also come with either it or Vi installed by default, so it’s ubiquitous.

Vim has a reputation for being difficult to use, but I don’t really think that’s the case. It’s just that it’s very different to use if you’re coming from most modern editors. Once you get accustomed to it, I’ve found it to be perhaps the most easy for editing text effectively. Plus, it has a rich plugin ecosystem for things like syntax highlighting and support. I use Pathogen as my plugin manager for Vim.

If you want to know my Vim config, this is my typical .vimrc file:

execute pathogen#infect()
set hlsearch ignorecase smartcase incsearch relativenumber ruler
set laststatus=2 tabstop=4 shiftwidth=4 expandtab notitle
syntax on
filetype plugin indent on

htop

htop is a super handy, interactive utility for seeing what the heck is happening in a Linux system. Think of it like the terminal version of the Windows Task Manager. It gives you a nice breakdown of CPU, RAM, and swap usage along with a listing of processes and what each is doing. It also offers and easy way to adjust the nice level of particularly important or greedy processes. It’s an enhancement over the older top utility.

Screenshot from 2019-05-06 17-32-51.png

tmux

tmux is terminal multiplexer, hence the name. If you have no clue what that means, it allows you to take a single terminal and divide it up into multiple virtual terminals. This lets easily have multiple terminals on the screen at the same time with different information on them without having to flip between tabs. Here’s a sample:

Screenshot from 2019-05-06 18-16-41.png

In a single terminal window I’ve got Vim open with some simple Go code on the left pane. The right side has two panes; the top pane has Cowsay running while the bottom pane I just used to install Cowsay. While not exact useful in the scenario I set up for this screenshot, it can be really handy for doing something like writing a script in one pane and having a second, smaller pane to the side or top of running it periodically without ever needing to close the file.

The other super handy part of tmux is that you can keep a persistent session going on a remote system without staying connected to it. I can SSH to a server, open tmux, connect to an IRC server, and use it for however long I need. If I want to disconnect from SSH but keep my IRC session going, I can simply detach my current SSH session from tmux but leave tmux running. Then I can close my SSH session. When I later SSH into the system again, I can reattach to the existing tmux session and pick right back up where I left off.

PowerShell

Since I’ve mentioned code a few times in these examples after starting the post off by saying I don’t know what the hell I’m doing when it comes to writing code, PowerShell is the one exception. I basically live in a PowerShell window for work, using it for both my day-to-day management at the CLI and in scripts that I use to automate my work… because work smart, not hard, right? I feel decently proficient with PowerShell, and I’m excited that version 6 is now open source! It’s nice to be able to use the same scripting language and commands to manage Windows servers at work that I use to manage my Linux servers at home.

I haven’t posted anything new in a while (I’ve written tons of stuff but just haven’t thought to post it), but you can see some of my sample PowerShell scripts over on GitHub. I’ve also posted some /r/DailyProgrammer challenges as Gists.

Hugo

Hugo is a static site generator. The concept is that instead of needing a CMS (content management system… think something like WordPress) to manage posts, pagination, design, etc. on a website that you can instead do it all via plaintext. Hugo allows you to have HTML templates, CSS, and then posts that are authored as Markdown. When you make a new post or change the site in any way, you can recompile your site which is then output as simple HTML and CSS that you can throw onto a web server. New post? Recompile the site and just move the files. Hugo worries about things like how many posts there should be on a page and will adjust it all for you. Need to change information in your header? Just change it one time in your template file and then recompile; there’s no need to use sed through every page and change each of them.

There are plenty of other static site generators out there (Jekyll is a popular one), but I’ve found that Hugo is by far the fastest. When your site starts to get large with a lot of pages to parse and generate, generators like Jekyll — which is written in Ruby, an interpreted language — can start to bog down. Hugo is written in Go; it’s literally one binary, and that allows it to be super speedy even when your site is large.

NetHack

To end the post on a fun note, NetHack is an incredible video game. It’s easy to look at it and assume that it’s a simplistic, basic game. It runs in a terminal (though variants with tiles and graphics do exist), and everything in the game is represented as an ASCII symbol. Your character? The @ symbol. A kobold? The letter k. The game is also crazy old… it was released in 1987. Here’s what it looks like:

Screenshot from 2019-05-06 18-38-44.png

It’s a fantasy game so the whole point is to hack-and-slash your way through a procedurally generated dungeon, meaning no two games are the same since each level is random; on top of this is the fact that there are more classes, races, and mechanics than most modern games have.

Even better is that the game is still being updated. The latest commit on their GitHub repo was yesterday. Pretty nuts. The game is also a marvel of what’s possible in the C programming language. Imagine making something like this without even being able to use objects.

NetHack exists for every operating system on the planet, but if you don’t want to bother with installing it you don’t have to. You can instead just hop on the alt.org NetHack server.

There are, of course, tons of other open source applications I use on the regular — I didn’t even bother getting into operating systems — but these are the ones I use the most frequently and enjoy the most. The most important thing, though… is to stay pink!