Work

Fixed: ScreenConnect / Control missing from Labtech / Automate

Automate screenshot

For the past two days my Automate window was missing all of the Screenconnect plugins that allow one click remote access to client machines. Both the one that shows at the top of the computer list and also when the machine window is launched. (Screenshot below shows how it should look)

Screenshot showing the control icon in Automate for computers

A reinstall of the software (including renaming the left over Labtech files in Program files and Program Data after removing the software) did not fix the issue.

However, reviewing the C:\ProgramData\LabTech Client\Logs\yyyymmdd_LTcErrors.txt showed lots of plugin exceptions including the following:-

An attempt was made to load an assembly from a network location which would have caused the assembly to be sandboxed in previous versions of the .NET Framework. This release of the .NET Framework does not enable CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly, please enable the loadFromRemoteSources switch. See http://go.microsoft.com/fwlink/?LinkId=155569 for more information

Following that link provided the hint that loadFromRemoteSources needs to be enabled.

Editing “C:\Program Files (x86)\LabTech Client\LTClient.exe.config” and adding <loadFromRemoteSources enabled=”true”/> just before the /runtime> line, Automate now includes the control button.

LTClient config file showing the loadfromremotesources element

Fixed: Lastpass seems to randomly add incorrect data to Forms.

We use a web based documentation system at work and have had a couple of instances where data for companies (ie Company X) seems to have been randomly edited in forms to include data from another form (ie Company Y) in the system. In a form that had a username, password, url and notes field we discovered that a tech could go in and edit the notes (and only the notes field) and without realising it, the username and password were also being updated in the form. The tech would hit save and now the saved password was incorrect.
Thankfully the documentation system has revision histories to allow us to revert back to the previous settings. but it is still a painful process to go back and review recent changes to see which ones were genuine edits and which were changed incorrectly.

We initially blamed it on LastPass filling out data as the issue would not occur if we disabled LastPass, however a search in LastPass would not return the data that was being added to the form. It took us a while to track down, but Chris, one of our techs worked out what was going on.

Sample lastpass password screen with extra field button highlighted

LastPass has additional fields that don’t show up when you browse (and apparently search) and the data from these extra fields were automatically being filled in for some reason. Click the wrench, highlighted in the above screenshot to see the extra hidden fields.

Our solution was to delete these extra fields, save the record in LastPass and we no longer have LastPass corrupting our data.

Happy Anniversary Absoblogginlutely!

16 years ago today I registered Absoblogginlutely.net and started to blog at this location. I totally missed the fact that back in March, helsby.net became 20 years old, a domain that I registered as an early birthday present to myself and is now used as my main email service.
This means I’ve been blogging on or off for about 20 years – how time flies!
Unfortunately I’ve not been updating this blog as often as I’d like as a lot of the tweaks and discoveries that I would normally blog about have become more work related and therefore more confidential.
However I would like to get back into the habit of documenting more so watch this space.
I’m heading to the Columbus Infosec Summit on Thursday and Friday this week which has always been full of interesting talks and demonstrations. It is sold out, but the twitter tag is .

Paula Januszkiewicz is one of the keynotes this year and her presentations are always valuable with a lot of takeaways and simultaneously manages to impress and scare me with the state of IT Security nowadays.

Fixed – Screenconnect blocked by Windows Smartscreen

Due to an expired code sign certificate, the version of Screenconnect that is launched from Connectwise Automate (aka Labtech) fails to run on 2 of my Windows 10 machines but works fine on the rest of the machines. The error message “Your administrator has blocked this application because it potentially poses a security risk to your computer”. The ones that fail are running Windows 1809 and 1903 so I suspect that there is some of the new features of SmartScreen are enabled and older versions do not have these settings.

Your administrator has blocked this application because it potentially poses a security risk to your computer

Checking out the file used for Screenconnect, I saw that the certificate used to sign the exe file expired on February 1st this year, but I’m not sure why my machines suddenly started to refuse to run it the last few days of March.

The Screenconnect.WindowsClient.exe is downloaded to a random subdirectory of appdata\local\apps\2.0 so I recommend you navigate to this directory and then search for *.exe and check the correct screenconnect file as per the screenshot below which shows the certificate expiring on the 1st February

ScreenConnect certificate expiry dates

After searching around and contacting Connectwise Support they advised me this would be fixed in an upcoming version. In the meantime setting the registry value HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft.NETFramework\Security\TrustManager\PromptingLevel\Internet to a string type of Enabled will allow the ClickOnce application to popup and this allows the dialog box to give an option as to whether the file should be run or not (the previous setting was Disabled). This then allows the user to select yes to install and run the file overriding the invalid SSL certificate.

Obviously this is not a great idea but it does allow you to run Screenconnect from within the Automate window. (The other alternative is to use the Screenconnect website itself to connect).

Fixed: PDF’s will not load over insecure http but will on secure https (if you have a Meraki firewall)

Had a weird issue this morning where pdf files that were served over http were getting blocked and would not load. Some sites also have https so we were able to just change the url to https and the file would then download.

After checking various browser settings I checked the Meraki firewall. By Disabling the Advanced Malware Protection (AMP) under Security/Threat Protection the files were downloaded successfully.
Instead of leaving AMP off, I then put a whitelist url of http://*.pdf and now pdf files load successfully with AMP still protecting the network.

Whitelisting pdf files in Meraki
Meraki pdf whitelisting

The frustrating thing is that AMP does not seem to be logged anywhere so it was not obvious that this was the cause of the problem. It also turns out that this has been a problem with AMP in the past as this 2 year old thread on Reddit shows!

Fixed: Scheduled automation task in Azure to start virtual machines does not complete successfully.

I use Azure’s automation accounts to stop and start virtual machines when they are not needed. One of our machines is used to host backups and is only used for a limited amount of time during the day when the backups run so it is shut down automatically after the backups have completed. 

Recently I ran some cleanup on my Azure subscription to get rid of some old test machines and in doing so removed one of the machines that was listed in an exclusion list of machines that should not be started or stopped (as I did not want this obsolete test machine starting on a daily basis).

Navigating to the Dashboard / Automation Accounts / Jobname / Scheduled Job/All logs shows that there is an invalid computer name.

Navigating through to Automation Account logs

It turns out that removing a machine from the Azure inventory but still keeping it in the VM’s exclude list actually stops the entire script from running.  Short term fix is to remove it from VM’s Exclude list and the machines start to run again.  The long term fix (one day) is to continue if the Exclude list contains invalid servers.

Powershell oneliner to check network connections used on current machine based on Mike Robin’s blog post

Mike Robins had a nice tip yesterday about using powershell to see what your system is talking to and I thought I would tweak it slightly to potentially make it even more useful.

When I ran the command on my machine it took a while to run and I also thought it would be nice to tweak it so that the machine does a reverse dns lookup to retrieve the host names that the system is talking to. This might provide an indication of whether the connection is good or not.

I saved the output of the command to a variable so if I need to tweak the display output I can do so easily without running the script again.

$a=Get-NetTCPConnection -State Established | `
Select-Object -Property LocalPort, RemoteAddress, RemotePort, State,`
@{name='Process';expression={(Get-Process -Id $_.OwningProcess).Name}}, `
@{name='fqdn';expression={([System.Net.Dns]::GetHostByAddress($_.Remoteaddress).Hostname)}},`
 CreationTime
$a

This is a very quick and dirty hack and takes ages to run on my computer. It probably doesn’t help that I have a ton of chrome tabs open which will require a lot of dns lookups and several of them are the same host but this method will lookup them all up individually. IP  and dns lookup on active network connections

 

Yes, I split this ‘one-liner’ into multiple lines to make it easier to read on the screen but if you have to do that, then it’s not really a one-liner and even more so if you are unlikely to remember it.

Office365 Exchange Control Panel now has command logging for admins.

Help Button, Show command logging.One of my annoyances with Office365 administration tasks was that I could make changes to the interface but had no idea what commands were being run behind the scenes. This made creating scripts a frustrating trial and error attempt at finding the correct verbs to run.
However, this morning I stumbled under the Help/Show Command Logging option in the admin panel. This is similar to the Show command output that was available in the Exchange 2010 admin console that I used extensively in the good old on-premise days.

I have no idea how long this has been here but it really made my day.

Now if only the rest of the Office365 admin panels had the same functionality.

Fixed: Unmountable Boot Volume error with Windows Server 2016 and Storagecraft’s SPX

BSOD imageWe’ve been tracking down issues with Windows Server 2016 on a multitude of servers this week where the servers will reboot and come back with Unmountable Boot Volume which is a pretty nasty experience for oncall. So far we’ve mainly seen it on Domain Controllers but also on a Hyper-V server. The solution is typically to do a last known good boot on the machine and then try to work out what has changed on the server and needs redoing. So far we’ve had issues with duplicate servers in Webroot and Automate along with a couple of server functions not working correctly.

Initially we thought it was a problem with Windows Updates, but it seems that the culprit is Storagecraft’s SPX version 6.7.4
The solution is either to downgrade to version 6.5 or get a patch for 6.7.4 that fixes this issue.

Download location for SPX 6.5.2:

For 6.7.4, You will need to get the patched stcvsm.sys  from Storagecraft and then apply these instructions.

Patch is a very manual process. New version of the stcvsm.sys driver is 2.2.73.0.36
1. Install SPX 6.7.2:
2. Do NOT reboot
3. Rename %windir%\system32\drivers\stcvsm.sys to %windir%\system32\drivers\stcvsm-rtm.sys
4. Copy the 2.2.73 driver to %windir%\system32\drivers. Be sure to select the correct ‘bitness’.
5. Reboot

It’s been very frustrating to have gone through this issue without any notification of this pretty serious bug from #Storagecraft

Edit: Today I discovered that Storagecraft now have a more detailed knowledge base article about resolving Inaccessible Boot Device after upgrade to 6.7.x. Judging from the comments I’ve had here, I’m not the only one who has had this issue and it still keeps happening for some users.