This is a slim down version of our Recovery & OS Uninstall Solutions that we will be presenting at MMS 2019, and will blog the fancy version then.
What this does: Restores the CM Client to a working state after a failed upgrade (Rollback) or if the system performs a Revert from New Build to Older Build after successful upgrade (OS Uninstall).
What happens to the CM Client during those Scenarios? Bad things man, bad things. Client is in provisioning mode, and services are disabled, it also returns to a place in time that it thinks it is running the upgrade task sequence.
Current MS Bugs… SetupRollback.cmd I’ve personally never been able to get this to run during a rollback, and MS has confirmed that there is a bug where it does not run during OS Uninstall.
My original plan was to tie into setuprollback.cmd, but without that, I’ve had to rely on creating some scheduled tasks that call powershell script that does the remediation.
So what does this slim down version do?
Before Upgrade, you have it copy PowerShell Scripts Local, and Create the Scheduled Task
- The Schedule task, Runs Daily and on Boot, calls PowerShell Script that Remediates issues.
- PowerShell checks for Rollback Key (HKLM:\SYSTEM\Setup\Rollback) then runs
- Removes from Provisioning Mode
- Run CCMEval to fix disabled Services
- Resets the Task Sequence so that Software Center is usable again.
- It also checks for the Windows 10 Setup Engine and kills that.**
- Copies logs to Server (File Share)
** In many of my tests, I found that due to the place the "Snap Shot" it taken by Windows, when the machine rolls back, it is in the middle of the Task Sequence, on the Step "Upgrade Windows", which calls the Setup Engine. After a rollback, and you fix the CM Client, it thinks it is running a TS and tries to pickup where it left off... triggering the Setup Engine... if you don't do anything, it will actually run the Windows 10 Upgrade, and perhaps the issue that broke the upgrade the first time is still present and now you're in a upgrade / fail loop. In image below, if you look closely at the logging I've added, you see that it actually saw Windows 10 Setup Engine was running, and it then successfully killed it. You can see the extra lines of code in the Remediation Script... do a search for "SetupHost"
1902 removes computer from Provisioning Mode within 24 hours.. which is great, but that doesn't get you nearly far enough. If you have a client health script running, it should fix the client, however it will NOT kill the Task Sequence or Prevent the accidental relaunch of the Setup Engine... Plus it will take 1 to 2 days before you'd even be aware of the issue. My script will resolve these issues and get you back up and running in 10 - 15 minutes after the failure.
UserVoice: OSUninstall - Make ConfigMgr Aware of going back to previous builds
OK... to the Solution I've built (Stand alone Version)
In Old Build (1803)
So now that all the “Stuff” is installed on the “old” build, you’re ready to upgrade, go ahead and run the upgrade to the new build. Note, you can add this into your upgrade TS. At MMS we’ll present how we add this into our Upgrade TS, before the Upgrade Step, and how we clean it up after the upgrade step.
For now, this will work for you to make sure you’re protected if someone fails the upgrade or if someone chooses to OS Uninstall (Go back to previous version)
Image after Rollback … Rollback Registry Key is present, you can see the script logging, and software center is back to working. It also shows the Upgrade TS as Failed, instead of Installing. Everything is back up and working, so you could then try the upgrade again.
Application:
Content Tab: Location where you place the 3 Files:
Install Command: "LoadOSRollbackRecovery.cmd"
Uninstall: powershell.exe -command "Unregister-ScheduledTask -TaskName OSRollbackRecovery -Confirm:$false"
Detection: File %Windir%\System32\Tasks OSRollbackRecovery (AKA The Name of the Scheduled Task)
LoadOSRollbackRecovery.cmd
xcopy *.ps1 %programdata%\OSRollBackRecovery\ /I /F /Y schtasks.exe /ru "SYSTEM" /Create /XML OSRollbackRecovery.xml /TN "OSRollbackRecovery" /F
OSRollbackRecovery.xml
<?xml version="1.0" encoding="UTF-16"?>
<Task version="1.2" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">
<RegistrationInfo>
<Date>2012-05-11T13:54:44.7469878</Date>
<Author>@GWBLOK</Author>
<URI>\OSRollbackRecovery</URI>
</RegistrationInfo>
<Triggers>
<BootTrigger>
<Enabled>true</Enabled>
</BootTrigger>
<CalendarTrigger>
<StartBoundary>2019-02-20T11:09:36</StartBoundary>
<ExecutionTimeLimit>PT2H</ExecutionTimeLimit>
<Enabled>true</Enabled>
<RandomDelay>PT8H</RandomDelay>
<ScheduleByDay>
<DaysInterval>1</DaysInterval>
</ScheduleByDay>
</CalendarTrigger>
</Triggers>
<Principals>
<Principal id="Author">
<UserId>S-1-5-18</UserId>
<RunLevel>HighestAvailable</RunLevel>
</Principal>
</Principals>
<Settings>
<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>
<DisallowStartIfOnBatteries>false</DisallowStartIfOnBatteries>
<StopIfGoingOnBatteries>false</StopIfGoingOnBatteries>
<AllowHardTerminate>false</AllowHardTerminate>
<StartWhenAvailable>false</StartWhenAvailable>
<RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable>
<IdleSettings>
<StopOnIdleEnd>true</StopOnIdleEnd>
<RestartOnIdle>false</RestartOnIdle>
</IdleSettings>
<AllowStartOnDemand>true</AllowStartOnDemand>
<Enabled>true</Enabled>
<Hidden>true</Hidden>
<RunOnlyIfIdle>false</RunOnlyIfIdle>
<WakeToRun>false</WakeToRun>
<ExecutionTimeLimit>PT0S</ExecutionTimeLimit>
<Priority>7</Priority>
</Settings>
<Actions Context="Author">
<Exec>
<Command>powershell.exe</Command>
<Arguments>-executionpolicy bypass -file "C:\ProgramData\OSRollBackRecovery\RollBackRecoveryStandAlone.ps1"</Arguments>
</Exec>
</Actions>
</Task>
RollBackRecoveryStandAlone.ps1
<#
RollBack (Failed Upgrade) Remediation script, this is triggered by a scheduled task that you should set to run daily / start up.
It will check to see if a rollback has happened and then do the required actions to fix the CM Client.
This is the Standalone version, which removes reporting, removes the scheduled tasks for updating Legal Text, but adds support for OSUninstall. (Phase 5)
Only Backs up logs to Server is Phase not equals 5.
#>
$RegistryRollBack = "HKLM:\SYSTEM\Setup\Rollback"
$RegistryTemp = "HKLM:\SOFTWARE\RollBack"
$LogFile = "C:\Windows\ccm\Logs\RollBackRecovery.log"
$ScriptName = $MyInvocation.MyCommand.Name
[string[]] $Path = @(
"$env:Systemdrive\`$WINDOWS.~BT\Sources\Panther"
"$env:Systemdrive\`$WINDOWS.~BT\Sources\Rollback"
"$env:SystemRoot\Panther"
"$env:SystemRoot\SysWOW64\PKG_LOGS"
"$env:SystemRoot\CCM\Logs"
)
[string] $TargetRoot = '\\src\Logs$'
[string] $LogID = "RollBack\$env:ComputerName"
[string[]] $Exclude = @( '*.exe','*.wim','*.dll','*.ttf','*.mui' )
[switch] $recurse
[switch] $SkipZip
function Test-RegistryValue {
param (
[parameter(Mandatory=$true)]
[ValidateNotNullOrEmpty()]$Path,
[parameter(Mandatory=$true)]
[ValidateNotNullOrEmpty()]$Value
)
try {
Get-ItemProperty -Path $Path | Select-Object -ExpandProperty $Value -ErrorAction Stop | Out-Null
return $true
}
catch {
return $false
}
}
#region: CMTraceLog Function formats logging in CMTrace style
function CMTraceLog {
[CmdletBinding()]
Param (
[Parameter(Mandatory=$false)]
$Message,
[Parameter(Mandatory=$false)]
$ErrorMessage,
[Parameter(Mandatory=$false)]
$Component = "RollBackRecovery",
[Parameter(Mandatory=$false)]
[int]$Type,
[Parameter(Mandatory=$true)]
$LogFile
)
<#
Type: 1 = Normal, 2 = Warning (yellow), 3 = Error (red)
#>
$Time = Get-Date -Format "HH:mm:ss.ffffff"
$Date = Get-Date -Format "MM-dd-yyyy"
if ($ErrorMessage -ne $null) {$Type = 3}
if ($Component -eq $null) {$Component = " "}
if ($Type -eq $null) {$Type = 1}
$LogMessage = "<![LOG[$Message $ErrorMessage" + "]LOG]!><time=`"$Time`" date=`"$Date`" component=`"$Component`" context=`"`" type=`"$Type`" thread=`"`" file=`"`">"
$LogMessage | Out-File -Append -Encoding UTF8 -FilePath $LogFile
}
function Disable-ProvMode
{
if ((Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode') -eq 'true')
{
$ProvMode = Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode' -ErrorAction SilentlyContinue
CMTraceLog -Message "ProvMode Status: $ProvMode" -Type 3 -LogFile $LogFile
if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "ProvMode Status: $ProvMode" -Type 3 -ServerLogFile $ServerLogFile}
CMTraceLog -Message "Removing Machine From Provisioning Mode and wait 30 seconds" -Type 2 -LogFile $LogFile
if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Removing Machine From Provisioning Mode and wait 30 seconds" -Type 2 -ServerLogFile $ServerLogFile}
Invoke-WmiMethod -Namespace root\CCM -Class SMS_Client -Name SetClientProvisioningMode -ArgumentList $false
Start-Sleep -Seconds 30
$ProvMode = Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode' -ErrorAction SilentlyContinue
if ($provmode -eq "True")
{
CMTraceLog -Message "ProvMode Status: $ProvMode" -Type 3 -LogFile $LogFile
if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "ProvMode Status: $ProvMode" -Type 3 -ServerLogFile $ServerLogFile}
CMTraceLog -Message "Removing Machine From Provisioning Mode" -Type 2 -LogFile $LogFile
if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Removing Machine From Provisioning Mode" -Type 2 -ServerLogFile $ServerLogFile}
Invoke-WmiMethod -Namespace root\CCM -Class SMS_Client -Name SetClientProvisioningMode -ArgumentList $false
}
Else
{
$ProvMode = Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode' -ErrorAction SilentlyContinue
CMTraceLog -Message "ProvMode Status: $ProvMode" -Type 1 -LogFile $LogFile
if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "ProvMode Status: $ProvMode" -Type 1 -ServerLogFile $ServerLogFile}
}
}
Else
{
$ProvMode = Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode' -ErrorAction SilentlyContinue
Write-Host "ProvMode Status: $ProvMode" -ForegroundColor Green
CMTraceLog -Message "ProvMode Status: $ProvMode" -Type 1 -LogFile $LogFile
if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "ProvMode Status: $ProvMode" -Type 1 -ServerLogFile $ServerLogFile}
}
}
#Create Function to Reset TS if running
function Reset-TaskSequence
{
Write-host "Starting Resetting CM Services to clear out TS" -ForegroundColor Yellow
CMTraceLog -Message "Starting Resetting CM Services to clear out TS - Takes about 3 minutes" -Type 2 -LogFile $LogFile
#if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Resetting CM Services to clear out TS" -Type 2 -ServerLogFile $ServerLogFile}
Set-Service smstsmgr -StartupType manual
Start-Service smstsmgr
CMTraceLog -Message "Stopping the CCMExec & TSManager Services (10 Seconds)" -Type 1 -LogFile $LogFile
if ((Get-Process CcmExec -ea SilentlyContinue) -ne $Null) {Get-Process CcmExec | Stop-Process -Force}
#stop-service ccmexec
if ((Get-Process TSManager -ea SilentlyContinue) -ne $Null) {Get-Process TSManager| Stop-Process -Force}
#Stop-Service smstsmgr
Start-Sleep -Seconds 5
CMTraceLog -Message "Starting the CCMExec & TSManager Services (30 Seconds)" -Type 1 -LogFile $LogFile
Start-Service ccmexec
Start-Sleep -Seconds 5
Start-Service smstsmgr
Start-Sleep -Seconds 20
CMTraceLog -Message "Stopping the CCMExec & TSManager Services (40 Seconds)" -Type 1 -LogFile $LogFile
if ((Get-Process TSManager -ea SilentlyContinue) -ne $Null) {Get-Process TSManager| Stop-Process -Force}
if ((Get-Process TSServiceUI -ea SilentlyContinue) -ne $Null) {Get-Process TSServiceUI | Stop-Process -Force}
Start-Sleep -Seconds 20
if ((Get-Process CcmExec -ea SilentlyContinue) -ne $Null) {Get-Process CcmExec | Stop-Process -Force}
if ((Get-Process TSServiceUI -ea SilentlyContinue) -ne $Null) {Get-Process TSServiceUI | Stop-Process -Force}
Start-Sleep -Seconds 15
CMTraceLog -Message "Starting the CCMExec Service (60 Seconds)" -Type 1 -LogFile $LogFile
Start-Service ccmexec
start-sleep -Seconds 60
CMTraceLog -Message "Triggering Machine Policy Updates" -Type 1 -LogFile $LogFile
Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000021}"
Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000022}"
#This looks for the Windows 10 Upgrade Procesa and Stops it, making sure it doesn't accidentally upgrade in an uncontrolled fassion.
if ((Get-Process "SetupHost" -ea SilentlyContinue) -eq $null){$SetupRunning = "False"}
Else
{
$SetupRunning = "True"
write-host "Setup Running - Stopping Now" -ForegroundColor Yellow
CMTraceLog -Message "Setup Running - Stopping Now" -Type 2 -LogFile $LogFile
#if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Setup Running - Stopping Now" -Type 2 -ServerLogFile $ServerLogFile}
Get-Process "SetupHost"| Stop-Process -Force
if ((Get-Process TSServiceUI -ea SilentlyContinue) -ne $Null) {Get-Process TSServiceUI | Stop-Process -Force}
start-sleep -Seconds 30
if ((Get-Process "SetupHost" -ea SilentlyContinue) -eq $null)
{$SetupRunning = "False"}
Else
{$SetupRunning = "True"
write-host "Setup Running - Stopping Now" -ForegroundColor Yellow
CMTraceLog -Message "Setup Running - Stopping Now" -Type 2 -LogFile $LogFile
#if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Setup Running - Stopping Now" -Type 2 -ServerLogFile $ServerLogFile}
Get-Process "SetupHost"| Stop-Process -Force
}
}
CMTraceLog -Message "Finished Resetting CM Services to clear out TS" -Type 2 -LogFile $LogFile
}
#Check for Rollback Key and run section if Rollback detected
if (Test-Path "$RegistryRollBack")
{
if ((Test-Path $Registrytemp) -ne $True -or (Get-ItemPropertyValue -Path "$RegistryTemp" -Name "OSRollbackRan") -eq "0")
{
CMTraceLog -Message "---Starting $ScriptName Script---" -Type 1 -LogFile $LogFile
#Set OSRollBackRan key to 0, to know where in the script it was if a reboot should occur
New-Item -Path $RegistryTemp –Force
Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "0" -Force
#Force the default Windows LockScreen images to be the actual LockScreen Image. Update for your envirnment.
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Policies\Microsoft\Windows\Personalization' -Name LockScreenImage -Value "C:\windows\Web\Screen\img100.jpg" -Force
Stop-Process -Name winlogon -Force -Verbose
CMTraceLog -Message "Starting CCM Disable ProvMode" -Type 1 -LogFile $LogFile
Disable-ProvMode
CMTraceLog -Message "Starting CCM Service & CCMEval" -Type 1 -LogFile $LogFile
Start-Process "C:\Windows\ccm\CcmEval.exe"
CMTraceLog -Message "Triggered CcmEval.exe" -Type 1 -LogFile $LogFile
#Set OSRollbackRan key to 1, to know where in the script it was if a reboot should occur
Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "1" -Force
}
if ((Get-ItemPropertyValue -Path "$RegistryTemp" -Name "OSRollbackRan") -eq "1")
{
CMTraceLog -Message "Waiting 5 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile
Start-Sleep -Seconds 60
CMTraceLog -Message "Waiting 4 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile
Start-Sleep -Seconds 60
CMTraceLog -Message "Waiting 3 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile
Start-Sleep -Seconds 60
CMTraceLog -Message "Waiting 2 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile
Start-Sleep -Seconds 60
CMTraceLog -Message "Waiting 1 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile
Start-Sleep -Seconds 60
Disable-ProvMode
Reset-TaskSequence
#Set OSRollbackRan key to 2, to know where in the script it was if a reboot should occur
Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "2" -Force
}
if ((Get-ItemPropertyValue -Path "$RegistryTemp" -Name "OSRollbackRan") -eq "2")
{
CMTraceLog -Message "Triggering CM Hardware Inventory" -Type 1 -LogFile $LogFile
Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000001}"
CMTraceLog -Message "Triggering CM Machine Policy Retrieval Cycle" -Type 1 -LogFile $LogFile
Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000021}"
CMTraceLog -Message "Triggering CM Machine Policy Evaluation Cycle" -Type 1 -LogFile $LogFile
Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000022}"
CMTraceLog -Message "Waiting 1 Minutes for Policy to Update" -Type 1 -LogFile $LogFile
Start-Sleep -Seconds 60
CMTraceLog -Message "Triggering CM Machine Policy Evaluation Cycle" -Type 1 -LogFile $LogFile
Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000113}"
#Added Phase of Failure to inventoried Values
#Set-ItemProperty -Path "$RegistryPathFull" -Name "RollBackPhase" -Value "$(Get-ItemPropertyValue -Path $RegistryRollBack -Name "Phase")" -Force
#Set-ItemProperty -Path "$RegistryPathFull" -Name "WaaS_Stage" -Value "Deployment_RollBack" -Force
#Set OSRollbackRan key to 3, to know where in the script it was if a reboot should occur, 3 basically means every time the script is triggered, it will do nothing, because it's already completed the required steps
Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "3" -Force
}
if ((Get-ItemPropertyValue -Path "$RegistryTemp" -Name "OSRollbackRan") -eq "3")
{
if ((Get-ItemPropertyValue -Path $RegistryRollBack -Name "Phase") -ne "5")
{
#Grab Logs and Backup To Server
CMTraceLog -Message "Backing Up Logs to Server" -Type 1 -LogFile $LogFile
CMTraceLog -Message "Location: $TargetRoot\$LogID" -Type 1 -LogFile $LogFile
#region Prepare Target
write-verbose "Log Archive Tool 1.0.<Version>"
write-verbose "Create Target $TargetRoot\$LogID"
new-item -itemtype Directory -Path $TargetRoot\$LogID -force -erroraction SilentlyContinue | out-null
$TagFile = "$TargetRoot\$LogID\$($LogID.Replace('\','_'))"
#endregion
#region Create temporary Store
$TempPath = [System.IO.Path]::GetTempFileName()
remove-item $TempPath
new-item -type directory -path $TempPath -force | out-null
foreach ( $Item in $Path ) {
$TmpTarget = (join-path $TempPath ( split-path -NoQualifier $Item ))
write-Verbose "COPy $Item to $TmpTarget"
copy-item -path $Item -Destination $TmpTarget -Force -Recurse -exclude $Exclude -ErrorAction SilentlyContinue
}
Compress-Archive -path "$TempPath\*" -DestinationPath "$TargetRoot\$LogID\$($LogID.Replace('\','_'))-$([datetime]::now.Tostring('s').Replace(':','-')).zip" -Force
remove-item $tempPath -Recurse -Force
#endregion
CMTraceLog -Message "Finished Backing Up Logs to Server" -Type 1 -LogFile $LogFile
}
#Set-ItemProperty -Path "$RegistryPathFull" -Name "RollBackLog" -Value "LogLocation: $TargetRoot\$LogID" -Force
Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "4" -Force
CMTraceLog -Message "---Exiting $ScriptName Script---" -Type 1 -LogFile $LogFile
Start-Process "C:\Windows\ccm\CcmEval.exe"
Exit
}
}
else {if ((Test-Path "$RegistryTemp") -eq 'True'){Remove-Item -Path "$RegistryTemp" -Force}}
Download (if Copy / Paste isn't your thing) : Rollback Recovery Stand Alone (758 downloads)
Posted on GARYTOWN.COM
2 thoughts on “Automated Client Recovery from Rollback or OS Uninstall”