Workaround to Dreamhost stopping the support of catchall emails by using Gmail and scripts

About 3 months ago, my hosting provider Dreamhost announced that it was stopping the support of catch-all email addresses claiming that this was to reduce the server load and improve antispam features.
Unfortunately this is a feature that I rely on heavily as I always provide unique email address’s to companies such as [email protected] or [email protected] so that I can filter out emails into different labels, blacklist a spammer and lastly see who has had their website compromised and leaked or sold my email address to someone else. I’ve had at least two email addresses, one of which was a local government agency, start to be used as a spam destination. Unfortunately it is frustrating when I report it to the company and they deny it has happened even though I’ve shown them headers of the email address after the initial contact to let them know they have been compromised. One of the companies I contacted denied they had been breached and I hadn’t even provided them the email address so I knew they hadn’t looked very hard!

Anyway….that is all now over as Dreamhost no longer provide this option.

The solution I am using is as follows:-

  • Use gmail as the primary interface for email (No change since I got gmail back in 2005ish)
  • Use gmail pop3 access to pull in the original catch all email and apply a “ca” label to the email (No change)
  • Use Google Sheets to analyze email and retrieve email address used
  • Create a list of incoming emails and destination email addresses
  • Add emails to Dreamhost control panel

If you wish to continue using the email addresses you can either add them as an alias or set up a forward for the email address. I was previously using the catchall mailbox and had gmail setup to retrieve the email on a regular basis.  The upside of this was that I didn’t have to do anything after setting up the pop account in Google. The downside is that I would have to wait for Google to poll the email account, or go to the accounts tab and pull the mail down manually if I was using a computer. The tablet/mobile access did not provide the ability to “check for mail now”

I had also setup gmail to label all mail retrieved from this catchall account with “ca”. This shows up in the google interface and can also be filtered/searched on.

This meant that I already have email in my gmail account. If you haven’t done that already and wish to use these instructions, then  set up gmail to pull in the email as a pop3 account.

Once you have all email coming into Google I then used a slight modification of Helge Klein’s spreadsheet that lists all email address’s for a single label. Use the provided link to copy the sheet into your own Google Drive account. Select the Tools menu, Edit script and review the script to make sure it’s not malicious.

Around line 45 will be the text

Replace this with

Yes I know the variable name doesn’t make sense, but this is the easiest way to quickly change the code. The neater solution is to then search and replace mailFrom with MailTo

Save the document and then use the text “ca newer:2017/12/31 older:2018/09/01” (without quotes) in B2in the sheets document. Obviously adjust the dates as appropriate.Sheets label to search for emails in 2018

Note that I found that if my date selection was too large the script would time out. Depending on how many emails you have in gmail, I would probably filter out a year at a time. The filter syntax you enter into the B2 field is the syntax you would use to search in gmail.

Once you have your filter, selectHK Scripts/Extract email addresses.

Hopefully all goes well and you now have a list of emails that were sent to your catch all address in a new tab.

The neat thing is that you can run the script multiple times and each search result will generate a new tab. This way you can run this periodically to capture mail received each month. The caveat is that you do need to keep emails with the ca label in your gmail for at least a month!

The screenshot below shows a few results from this search.

Note that I do have a couple of oddities in this result (which is why I included them). There are a couple of email address’s that go to my catchall address but the emails were not forwarded to my original domain. My gravatar email address is actually another gmail address forwarded to a catch all address. As the original TO field is not my domain it doesn’t look quite right. Some other oddities have included google groups targets or some other nefarious/badly formed bcc emails. These should be treated as an exception and handled manually (outside the scope of this document.

 

L:ist of catch all email addresses used

Grab the contents of column B and save to a new tab – this new tab will be used to split the email addresses into a format suitable to paste into Dreamhosts mail control panel.

I then did the following to parse the email address into a useable format. Note that this may not be the most efficient but it gives me the ability to easily verify each stage rather than debug a complicated excel formula.

The screenshot below shows what it should look like when completed. (The two shaded rows are the odd ones that I deal with manually)Spreadsheet of email addresses ready to paste into Dreamhost

Column A should be your original To field. The other cells are filled out as follows. Note you need the starting = sign to signify a calculated field.

D   =split(A1,“@”)

E – not used.

=” yourgooglealias+” & D1 & “@gmail.com”

Note there is a space at the start of G as this is used as the separator in the next step.  Yourgooglealias is the bit before the @ sign in your gmail address

=(substitute(substitute(A1& G1,“<“,“”),“>”,“”))

This replaces any chevrons that may be around the email address and then takes the original email address, adds a space and then adds the new gmail address where the alias now becomes +alias.

 

Finally, copy the contents of column K into dreamhosts control panel. https://panel.dreamhost.com/index.cgi?tree=mail.addresses&current_step=Index&next_step=Bulk should hopefully work but make sure you are pointing to the right domain!

Select Mail, your mail domain, Forwarding Only/Edit All.

Pasting emails into Dreamhost

Hit the save button and you are done.

Your email should be quickly delivered to gmail and still easily filtered out based on the to field.

The downside is that there are still some webforms out there that do not recognise the + character as valid in the email address. Spammers can also easily work out what your main email address is and send directly to that mailbox.

 

Please let me know if you have any questions or comments

 

Powershell oneliner to check network connections used on current machine based on Mike Robin’s blog post

Mike Robins had a nice tip yesterday about using powershell to see what your system is talking to and I thought I would tweak it slightly to potentially make it even more useful.

When I ran the command on my machine it took a while to run and I also thought it would be nice to tweak it so that the machine does a reverse dns lookup to retrieve the host names that the system is talking to. This might provide an indication of whether the connection is good or not.

I saved the output of the command to a variable so if I need to tweak the display output I can do so easily without running the script again.

This is a very quick and dirty hack and takes ages to run on my computer. It probably doesn’t help that I have a ton of chrome tabs open which will require a lot of dns lookups and several of them are the same host but this method will lookup them all up individually. IP  and dns lookup on active network connections

 

Yes, I split this ‘one-liner’ into multiple lines to make it easier to read on the screen but if you have to do that, then it’s not really a one-liner and even more so if you are unlikely to remember it.

Office365 Exchange Control Panel now has command logging for admins.

Help Button, Show command logging.One of my annoyances with Office365 administration tasks was that I could make changes to the interface but had no idea what commands were being run behind the scenes. This made creating scripts a frustrating trial and error attempt at finding the correct verbs to run.
However, this morning I stumbled under the Help/Show Command Logging option in the admin panel. This is similar to the Show command output that was available in the Exchange 2010 admin console that I used extensively in the good old on-premise days.

I have no idea how long this has been here but it really made my day.

Now if only the rest of the Office365 admin panels had the same functionality.

Book Review – Lies by T.M. Logan

LiesLies by T.M. Logan
My rating: 4 of 5 stars

T.M. Logan was a new author to me that I came across on Netgalley who gave me an arc version of the book for my review.
Overall I enjoyed the book but the first part seemed a bit tedious with all the dialog and very short chapters that seemed to interrupt the flow of the book.
However as the mystery and lies unfolded, the book got easier to read and the plot twists suck the reader deeper into the story.
I didn’t suspect the ending twist but it did seem plausible enough.
A good, reasonably quick summers read that would be perfect for the beach.

View all my reviews

Fixed: Unmountable Boot Volume error with Windows Server 2016 and Storagecraft’s SPX

BSOD imageWe’ve been tracking down issues with Windows Server 2016 on a multitude of servers this week where the servers will reboot and come back with Unmountable Boot Volume which is a pretty nasty experience for oncall. So far we’ve mainly seen it on Domain Controllers but also on a Hyper-V server. The solution is typically to do a last known good boot on the machine and then try to work out what has changed on the server and needs redoing. So far we’ve had issues with duplicate servers in Webroot and Automate along with a couple of server functions not working correctly.

Initially we thought it was a problem with Windows Updates, but it seems that the culprit is Storagecraft’s SPX version 6.7.4
The solution is either to downgrade to version 6.5 or get a patch for 6.7.4 that fixes this issue.

Download location for SPX 6.5.2:

For 6.7.4, You will need to get the patched stcvsm.sys  from Storagecraft and then apply these instructions.

Patch is a very manual process. New version of the stcvsm.sys driver is 2.2.73.0.36
1. Install SPX 6.7.2:
2. Do NOT reboot
3. Rename %windir%\system32\drivers\stcvsm.sys to %windir%\system32\drivers\stcvsm-rtm.sys
4. Copy the 2.2.73 driver to %windir%\system32\drivers. Be sure to select the correct ‘bitness’.
5. Reboot

 

It’s been very frustrating to have gone through this issue without any notification of this pretty serious bug from #Storagecraft

Book Review: The Book Of Joe

The Book of Joe: The Life, Wit, and (Sometimes Accidental) Wisdom of Joe BidenThe Book of Joe: The Life, Wit, and (Sometimes Accidental) Wisdom of Joe Biden by Jeff Wilser
My rating: 4 of 5 stars

The last book I read in 2017 but I’ve been sick and unable to post to the blog until 2018!  The description of this book was interesting and I’ve been trying to alternate my reading between fiction and non fiction and as a Brit in the USA I basically know nothing about Joe Biden apart from his much meme’d bromance with Obama so I figured this might make an interesting read.

The book is a simple, short biography of Biden that focus’s on tips of wisdom from Biden along with the stories behind them. Covering a lot of episodes from being a kid through to life after the election, the reader is taken through several key moments in his life, along with a challenge to apply these tips to our lives.

Thankfully the book doesn’t get too political so is suitable for all to read – treat it as a biography rather than a political memoir.
The author is obviously a huge, huge fan of Biden and if Biden’s bromance with Obama never worked out, I think Jeff would be waiting in the wings!

Thanks to Blogging for Books for the review copy of this book and the opportunity to read a funny, inspiring book about a politician (not something I would ever expect to say!)

View all my reviews

How to work around “Thank you for updating to the latest version of Windows 10”

Getting “Thank you for updating to the latest version of Windows 10” message with an exit button rather than the option to upgrade to Windows 10 Fall Creators Update?

I’ve been waiting for the Fall Creators update to be available so that I can now start using the Files On Demand feature that allows you to see files from OneDrive that have not been synched down to the pc yet – a feature that used to be called placeholders. I saw it demoed at Ignite and this finally means Onedrive for Business can be useful for businesses now.

Anyway – In a classic sleepless early morning I checked Windows Update last night and this morning and it said that my computer was up to date.

Checking the Windows 10 download page, and then using the update now button downloads the upgrade tool but I then get “Thank you for updating to the latest version of windows 10” error message.

The trick is to download the  “create media tool”. Oddly enough, the first question this tool asks is if you want to update this machine or download the media iso to update other machines.  Pick the first option and away you go!

Installing Windows 10 Fall Creators Update

Now where did paint go? 😉

So what new features are you looking forward to using?

Prepping for #MSIgnite – 50% discount on exams taken at the conference.

I was eventually able to find a post on Uservoice that confirmed there was a discount on Microsoft exams if taken at MSIgnite. Pick the exam and register in the normal way, but when selecting the test center, choose Orlando, FL and then select the Ignite center. This will then give you a 50% discount on the exam that will show at the bottom of the invoice.

Microsoft exam discount

All the MSIgnite information is scattered all over the place so you really do need to keep an eye out on the forums and twitter to find out whats happening.

I’m not sure what exams I will take this year but would like to get at least one done whilst I am there.

You can check out the rest of my #MSignite posts here.

Prepping for #MSIgnite – Mobile apps

It appears that the #MSIgnite2017 mobile apps are ready for download now. The Android app is available here and the Apple app here.

I only downloaded the Android version of the app although I was a bit concerned about the authenticity of the app. I have not seen any official notification from Microsoft about the apps being available and the publisher of the application is Eventbase Technology and not Microsoft. The reviews were also pretty scathing saying that the logins did not work so all the signs were pointing to a phishing attack.Microsoft Ignite app on the Google play store. Would you trust this app?However, Eventbase Technology, Inc seem to do a lot of event apps for various people so it sounded like it may be legit. After the app was downloaded it did go to an official looking live.com signin page and as I have 2fa enabled on my account and my account password is different that passwords used anywhere else I didn’t feel too nervous about signing in.

Curiously the app has an option to create a Live Id as it’s just providing the standard Live Id login page but this functionality is kind of pointless as you need an id to sign up for MSIgnite in the first place.

Unlike the others, I was able to sign in successfully and the schedule that I’ve already setup online synchs down nicely to the app. As you can see from the image below it doesn’t help with overlapping schedules 😉 but you can see the ability to add/remove sessions to the schedule.Android Session Scheduler for #MSIgnite 2017

Interestingly, I still don’t see the keynotes listed in the session schedules. Anyone know when these are and why they would not be in the scheduler?

Prepping for #MSIgnite – External battery pack review

In part 2 of my prepping for #MSIgnite series (see also part 1 for Session plans) I thought I would document my thoughts for extending the battery life for my laptop(s). I’m still not sure whether to take my personal Surface Pro3 or the company laptop – a Lenovo P50S. The former is lighter, smaller and much easier to lug around the conference all day and with the touch screen and Onenote I will finally be able to use the full onenote experience of recording a talk whilst making notes in Onenote with the automatic audio bookmarking enabling me to go back in time for the audio at the point that I made the notes.  The latter has the bigger screen but that is basically it. Most of the work apps are browser based apart from outlook but I can use OWA for that too so really either machine can be used.

The downside is that neither of the machines have a great battery life. In recent usage at work, the Lenovo didn’t even make it through half a day before needing power and I know the Surface isn’t that great either. Note that neither of the computers currently use the battery saver as the screen brightness and performance normally needs to be at their brightness for day to day usage.

For the conference I will probably be using battery saver mode to eek out as much battery life as possible. I didn’t realise there were so many options for the battery when you hit start/battery in Windows 10

Options available in Win10 after Start / Battery
Start / Battery

As I didn’t want to run out of battery and I don’t think I will take both laptops with me each day on the conference I decided to get an external battery pack that will allow me to charge either of the laptops and also my phone (which seems to have a 6 hour lifespan too).

The FAA has rules that determine the size of the battery that can be taken on a plane and as I’m not driving down to Florida, this was one of the biggest factors in selecting a device (plus price of course) . The FAA regulations basically state that you can’t have anything that has a capacity of over 100wH in either carry on or checked luggage.  Unfortunately a lot of the power pack ratings on Amazon focus on the total capacity as the bigger the number the more impressive it sounds and they often do not state the wH which is what the FAA require.  The FAA does provide some calculations to obtain wH (divide the mA by 1000 and multiply by the volts, but this ends up with widely different numbers if you have different voltages for the battery pack.  However searching the questions or reviews for the word fly or plane will often show someone else asking if the device can be carried on the plane.

My initial battery choice was 50,000mAh as this would apparently charge the surface about 2-3 times based on one comment but this was way over the FAA allowance – however, batteries in the 25,000mAh seemed to be compliant so in the end I decided to get the Poweradd Pilot Pro2 23000mAh Power Bank for $89 from Amazon as this is 85wH and therefore under the limit. The rating is stamped on the back of the device to easily show security at the airport if needed. This device comes with tons of different tips for the various laptop inputs including the weird Lenovo tip that I would need. Unfortunately it did not come with the Surface adapter so that was an extra $9 to get the Threeeggs DC Plug Charging Cord Power Supply for Microsoft Surface Pro 4 (which is also compatible with the Surface Pro 3). This adapter has a nice generous 5′ cord to go between the laptop and the battery bank which makes it a lot easier to use on the go. Combine that with the 7′ long power supply (if you wanted to keep the battery charging) and you have the nice ability to keep connected and trip up lots of people – so be careful where you put those cords!

 

Size wise, the battery pack is just under half the size of a surface pro 3 at 4.75 by 7.25 (the surface is  11.5 by 8in) and 0.75 vs 05 in thick so should easily fit in your laptop carry case – see photo below for comparison.Surface Pro 3 and Poweradd powerbank

 

As far as the power is concerned, after about 15 minutes on charge with the battery, the power pack had depleted 8% and the laptop was up to 12% full from the 4% that was left on the laptop after I received the low battery warning. At this point in time, the time to charge to full capacity on the laptop was 5 hours and 1 minute. In comparison the time to charge using a mains charger was 2 hours and 2 minutes. Note that you have to wait a few seconds for the estimated full charge indicator to popup after entering the battery settings on the pc.Battery charging indicatorThe charge time seems to be pretty excessive but the good news is that you can still use the computer while the battery is being charged – there were a couple of reviews on other power packs that mentioned that the laptop would not actually charge whilst plugged in if the laptop was in use. One of the reviews stated you should switch the output of the power bank  to 15v if you want to use the Surface and charging. This power bank also goes up to 19v which will be required to charge the Lenovo P50S.

After about 50 minutes I was up to 26% charged, with an estimated 2 hours left. I closed the laptop and left it complete the charge. When I came back 3 hours later, the Surface was fully charged and I had 18% left on the battery pack. This will definitely give me enough power to extend the usage of the laptop all day at Ignite and the power pack is quick enough to recharge overnight ready for the next day. Obviously a power strip is also in my packing list for #MSIgnite

I’ll be running some comparative tests on the Lenovo P50s later to see how the battery does for this device but thought I would get another post published before it’s too late.

So far it looks like this Power Bank will do a good job for me at MSIgnite2017 – at under $100 with all the tips I’d ever need (and many more besides), this won’t break the bank and should let me stay connected all day.

Let me know if you’ve tried any other battery packs and if you have other tips for keeping connected whilst at the conference.