This is a slim down version of our Recovery & OS Uninstall Solutions that we will be presenting at MMS 2019, and will blog the fancy version then.
What this does: Restores the CM Client to a working state after a failed upgrade (Rollback) or if the system performs a Revert from New Build to Older Build after successful upgrade (OS Uninstall).
What happens to the CM Client during those Scenarios? Bad things man, bad things. Client is in provisioning mode, and services are disabled, it also returns to a place in time that it thinks it is running the upgrade task sequence.
Current MS Bugs… SetupRollback.cmd I’ve personally never been able to get this to run during a rollback, and MS has confirmed that there is a bug where it does not run during OS Uninstall.
My original plan was to tie into setuprollback.cmd, but without that, I’ve had to rely on creating some scheduled tasks that call powershell script that does the remediation.
So what does this slim down version do?
Before Upgrade, you have it copy PowerShell Scripts Local, and Create the Scheduled Task
- The Schedule task, Runs Daily and on Boot, calls PowerShell Script that Remediates issues.
- PowerShell checks for Rollback Key (HKLM:\SYSTEM\Setup\Rollback) then runs
- Removes from Provisioning Mode
- Run CCMEval to fix disabled Services
- Resets the Task Sequence so that Software Center is usable again.
- It also checks for the Windows 10 Setup Engine and kills that.**
- Copies logs to Server (File Share)
** In many of my tests, I found that due to the place the “Snap Shot” it taken by Windows, when the machine rolls back, it is in the middle of the Task Sequence, on the Step “Upgrade Windows”, which calls the Setup Engine. After a rollback, and you fix the CM Client, it thinks it is running a TS and tries to pickup where it left off… triggering the Setup Engine… if you don’t do anything, it will actually run the Windows 10 Upgrade, and perhaps the issue that broke the upgrade the first time is still present and now you’re in a upgrade / fail loop. In image below, if you look closely at the logging I’ve added, you see that it actually saw Windows 10 Setup Engine was running, and it then successfully killed it. You can see the extra lines of code in the Remediation Script… do a search for “SetupHost”
1902 removes computer from Provisioning Mode within 24 hours.. which is great, but that doesn’t get you nearly far enough. If you have a client health script running, it should fix the client, however it will NOT kill the Task Sequence or Prevent the accidental relaunch of the Setup Engine… Plus it will take 1 to 2 days before you’d even be aware of the issue. My script will resolve these issues and get you back up and running in 10 – 15 minutes after the failure.
UserVoice: OSUninstall – Make ConfigMgr Aware of going back to previous builds
OK… to the Solution I’ve built (Stand alone Version)
So now that all the “Stuff” is installed on the “old” build, you’re ready to upgrade, go ahead and run the upgrade to the new build. Note, you can add this into your upgrade TS. At MMS we’ll present how we add this into our Upgrade TS, before the Upgrade Step, and how we clean it up after the upgrade step.
For now, this will work for you to make sure you’re protected if someone fails the upgrade or if someone chooses to OS Uninstall (Go back to previous version)
Image after Rollback … Rollback Registry Key is present, you can see the script logging, and software center is back to working. It also shows the Upgrade TS as Failed, instead of Installing. Everything is back up and working, so you could then try the upgrade again.
Application:
Content Tab: Location where you place the 3 Files:
Install Command: “LoadOSRollbackRecovery.cmd”
Uninstall: powershell.exe -command “Unregister-ScheduledTask -TaskName OSRollbackRecovery -Confirm:$false”
Detection: File %Windir%\System32\Tasks OSRollbackRecovery (AKA The Name of the Scheduled Task)
LoadOSRollbackRecovery.cmd
xcopy *.ps1 %programdata%\OSRollBackRecovery\ /I /F /Y schtasks.exe /ru "SYSTEM" /Create /XML OSRollbackRecovery.xml /TN "OSRollbackRecovery" /F
OSRollbackRecovery.xml
<?xml version="1.0" encoding="UTF-16"?> <Task version="1.2" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task"> <RegistrationInfo> <Date>2012-05-11T13:54:44.7469878</Date> <Author>@GWBLOK</Author> <URI>\OSRollbackRecovery</URI> </RegistrationInfo> <Triggers> <BootTrigger> <Enabled>true</Enabled> </BootTrigger> <CalendarTrigger> <StartBoundary>2019-02-20T11:09:36</StartBoundary> <ExecutionTimeLimit>PT2H</ExecutionTimeLimit> <Enabled>true</Enabled> <RandomDelay>PT8H</RandomDelay> <ScheduleByDay> <DaysInterval>1</DaysInterval> </ScheduleByDay> </CalendarTrigger> </Triggers> <Principals> <Principal id="Author"> <UserId>S-1-5-18</UserId> <RunLevel>HighestAvailable</RunLevel> </Principal> </Principals> <Settings> <MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy> <DisallowStartIfOnBatteries>false</DisallowStartIfOnBatteries> <StopIfGoingOnBatteries>false</StopIfGoingOnBatteries> <AllowHardTerminate>false</AllowHardTerminate> <StartWhenAvailable>false</StartWhenAvailable> <RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable> <IdleSettings> <StopOnIdleEnd>true</StopOnIdleEnd> <RestartOnIdle>false</RestartOnIdle> </IdleSettings> <AllowStartOnDemand>true</AllowStartOnDemand> <Enabled>true</Enabled> <Hidden>true</Hidden> <RunOnlyIfIdle>false</RunOnlyIfIdle> <WakeToRun>false</WakeToRun> <ExecutionTimeLimit>PT0S</ExecutionTimeLimit> <Priority>7</Priority> </Settings> <Actions Context="Author"> <Exec> <Command>powershell.exe</Command> <Arguments>-executionpolicy bypass -file "C:\ProgramData\OSRollBackRecovery\RollBackRecoveryStandAlone.ps1"</Arguments> </Exec> </Actions> </Task>
RollBackRecoveryStandAlone.ps1
<# RollBack (Failed Upgrade) Remediation script, this is triggered by a scheduled task that you should set to run daily / start up. It will check to see if a rollback has happened and then do the required actions to fix the CM Client. This is the Standalone version, which removes reporting, removes the scheduled tasks for updating Legal Text, but adds support for OSUninstall. (Phase 5) Only Backs up logs to Server is Phase not equals 5. #> $RegistryRollBack = "HKLM:\SYSTEM\Setup\Rollback" $RegistryTemp = "HKLM:\SOFTWARE\RollBack" $LogFile = "C:\Windows\ccm\Logs\RollBackRecovery.log" $ScriptName = $MyInvocation.MyCommand.Name [string[]] $Path = @( "$env:Systemdrive\`$WINDOWS.~BT\Sources\Panther" "$env:Systemdrive\`$WINDOWS.~BT\Sources\Rollback" "$env:SystemRoot\Panther" "$env:SystemRoot\SysWOW64\PKG_LOGS" "$env:SystemRoot\CCM\Logs" ) [string] $TargetRoot = '\\src\Logs$' [string] $LogID = "RollBack\$env:ComputerName" [string[]] $Exclude = @( '*.exe','*.wim','*.dll','*.ttf','*.mui' ) [switch] $recurse [switch] $SkipZip function Test-RegistryValue { param ( [parameter(Mandatory=$true)] [ValidateNotNullOrEmpty()]$Path, [parameter(Mandatory=$true)] [ValidateNotNullOrEmpty()]$Value ) try { Get-ItemProperty -Path $Path | Select-Object -ExpandProperty $Value -ErrorAction Stop | Out-Null return $true } catch { return $false } } #region: CMTraceLog Function formats logging in CMTrace style function CMTraceLog { [CmdletBinding()] Param ( [Parameter(Mandatory=$false)] $Message, [Parameter(Mandatory=$false)] $ErrorMessage, [Parameter(Mandatory=$false)] $Component = "RollBackRecovery", [Parameter(Mandatory=$false)] [int]$Type, [Parameter(Mandatory=$true)] $LogFile ) <# Type: 1 = Normal, 2 = Warning (yellow), 3 = Error (red) #> $Time = Get-Date -Format "HH:mm:ss.ffffff" $Date = Get-Date -Format "MM-dd-yyyy" if ($ErrorMessage -ne $null) {$Type = 3} if ($Component -eq $null) {$Component = " "} if ($Type -eq $null) {$Type = 1} $LogMessage = "<![LOG[$Message $ErrorMessage" + "]LOG]!><time=`"$Time`" date=`"$Date`" component=`"$Component`" context=`"`" type=`"$Type`" thread=`"`" file=`"`">" $LogMessage | Out-File -Append -Encoding UTF8 -FilePath $LogFile } function Disable-ProvMode { if ((Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode') -eq 'true') { $ProvMode = Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode' -ErrorAction SilentlyContinue CMTraceLog -Message "ProvMode Status: $ProvMode" -Type 3 -LogFile $LogFile if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "ProvMode Status: $ProvMode" -Type 3 -ServerLogFile $ServerLogFile} CMTraceLog -Message "Removing Machine From Provisioning Mode and wait 30 seconds" -Type 2 -LogFile $LogFile if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Removing Machine From Provisioning Mode and wait 30 seconds" -Type 2 -ServerLogFile $ServerLogFile} Invoke-WmiMethod -Namespace root\CCM -Class SMS_Client -Name SetClientProvisioningMode -ArgumentList $false Start-Sleep -Seconds 30 $ProvMode = Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode' -ErrorAction SilentlyContinue if ($provmode -eq "True") { CMTraceLog -Message "ProvMode Status: $ProvMode" -Type 3 -LogFile $LogFile if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "ProvMode Status: $ProvMode" -Type 3 -ServerLogFile $ServerLogFile} CMTraceLog -Message "Removing Machine From Provisioning Mode" -Type 2 -LogFile $LogFile if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Removing Machine From Provisioning Mode" -Type 2 -ServerLogFile $ServerLogFile} Invoke-WmiMethod -Namespace root\CCM -Class SMS_Client -Name SetClientProvisioningMode -ArgumentList $false } Else { $ProvMode = Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode' -ErrorAction SilentlyContinue CMTraceLog -Message "ProvMode Status: $ProvMode" -Type 1 -LogFile $LogFile if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "ProvMode Status: $ProvMode" -Type 1 -ServerLogFile $ServerLogFile} } } Else { $ProvMode = Get-ItemPropertyValue 'HKLM:\SOFTWARE\Microsoft\CCM\CcmExec' 'ProvisioningMode' -ErrorAction SilentlyContinue Write-Host "ProvMode Status: $ProvMode" -ForegroundColor Green CMTraceLog -Message "ProvMode Status: $ProvMode" -Type 1 -LogFile $LogFile if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "ProvMode Status: $ProvMode" -Type 1 -ServerLogFile $ServerLogFile} } } #Create Function to Reset TS if running function Reset-TaskSequence { Write-host "Starting Resetting CM Services to clear out TS" -ForegroundColor Yellow CMTraceLog -Message "Starting Resetting CM Services to clear out TS - Takes about 3 minutes" -Type 2 -LogFile $LogFile #if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Resetting CM Services to clear out TS" -Type 2 -ServerLogFile $ServerLogFile} Set-Service smstsmgr -StartupType manual Start-Service smstsmgr CMTraceLog -Message "Stopping the CCMExec & TSManager Services (10 Seconds)" -Type 1 -LogFile $LogFile if ((Get-Process CcmExec -ea SilentlyContinue) -ne $Null) {Get-Process CcmExec | Stop-Process -Force} #stop-service ccmexec if ((Get-Process TSManager -ea SilentlyContinue) -ne $Null) {Get-Process TSManager| Stop-Process -Force} #Stop-Service smstsmgr Start-Sleep -Seconds 5 CMTraceLog -Message "Starting the CCMExec & TSManager Services (30 Seconds)" -Type 1 -LogFile $LogFile Start-Service ccmexec Start-Sleep -Seconds 5 Start-Service smstsmgr Start-Sleep -Seconds 20 CMTraceLog -Message "Stopping the CCMExec & TSManager Services (40 Seconds)" -Type 1 -LogFile $LogFile if ((Get-Process TSManager -ea SilentlyContinue) -ne $Null) {Get-Process TSManager| Stop-Process -Force} if ((Get-Process TSServiceUI -ea SilentlyContinue) -ne $Null) {Get-Process TSServiceUI | Stop-Process -Force} Start-Sleep -Seconds 20 if ((Get-Process CcmExec -ea SilentlyContinue) -ne $Null) {Get-Process CcmExec | Stop-Process -Force} if ((Get-Process TSServiceUI -ea SilentlyContinue) -ne $Null) {Get-Process TSServiceUI | Stop-Process -Force} Start-Sleep -Seconds 15 CMTraceLog -Message "Starting the CCMExec Service (60 Seconds)" -Type 1 -LogFile $LogFile Start-Service ccmexec start-sleep -Seconds 60 CMTraceLog -Message "Triggering Machine Policy Updates" -Type 1 -LogFile $LogFile Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000021}" Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000022}" #This looks for the Windows 10 Upgrade Procesa and Stops it, making sure it doesn't accidentally upgrade in an uncontrolled fassion. if ((Get-Process "SetupHost" -ea SilentlyContinue) -eq $null){$SetupRunning = "False"} Else { $SetupRunning = "True" write-host "Setup Running - Stopping Now" -ForegroundColor Yellow CMTraceLog -Message "Setup Running - Stopping Now" -Type 2 -LogFile $LogFile #if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Setup Running - Stopping Now" -Type 2 -ServerLogFile $ServerLogFile} Get-Process "SetupHost"| Stop-Process -Force if ((Get-Process TSServiceUI -ea SilentlyContinue) -ne $Null) {Get-Process TSServiceUI | Stop-Process -Force} start-sleep -Seconds 30 if ((Get-Process "SetupHost" -ea SilentlyContinue) -eq $null) {$SetupRunning = "False"} Else {$SetupRunning = "True" write-host "Setup Running - Stopping Now" -ForegroundColor Yellow CMTraceLog -Message "Setup Running - Stopping Now" -Type 2 -LogFile $LogFile #if ($RunningAsSystem -eq "True"-and $ScriptLogging -eq "True"){CMTraceServerLog -Message "Setup Running - Stopping Now" -Type 2 -ServerLogFile $ServerLogFile} Get-Process "SetupHost"| Stop-Process -Force } } CMTraceLog -Message "Finished Resetting CM Services to clear out TS" -Type 2 -LogFile $LogFile } #Check for Rollback Key and run section if Rollback detected if (Test-Path "$RegistryRollBack") { if ((Test-Path $Registrytemp) -ne $True -or (Get-ItemPropertyValue -Path "$RegistryTemp" -Name "OSRollbackRan") -eq "0") { CMTraceLog -Message "---Starting $ScriptName Script---" -Type 1 -LogFile $LogFile #Set OSRollBackRan key to 0, to know where in the script it was if a reboot should occur New-Item -Path $RegistryTemp –Force Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "0" -Force #Force the default Windows LockScreen images to be the actual LockScreen Image. Update for your envirnment. Set-ItemProperty -Path 'HKLM:\SOFTWARE\Policies\Microsoft\Windows\Personalization' -Name LockScreenImage -Value "C:\windows\Web\Screen\img100.jpg" -Force Stop-Process -Name winlogon -Force -Verbose CMTraceLog -Message "Starting CCM Disable ProvMode" -Type 1 -LogFile $LogFile Disable-ProvMode CMTraceLog -Message "Starting CCM Service & CCMEval" -Type 1 -LogFile $LogFile Start-Process "C:\Windows\ccm\CcmEval.exe" CMTraceLog -Message "Triggered CcmEval.exe" -Type 1 -LogFile $LogFile #Set OSRollbackRan key to 1, to know where in the script it was if a reboot should occur Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "1" -Force } if ((Get-ItemPropertyValue -Path "$RegistryTemp" -Name "OSRollbackRan") -eq "1") { CMTraceLog -Message "Waiting 5 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile Start-Sleep -Seconds 60 CMTraceLog -Message "Waiting 4 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile Start-Sleep -Seconds 60 CMTraceLog -Message "Waiting 3 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile Start-Sleep -Seconds 60 CMTraceLog -Message "Waiting 2 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile Start-Sleep -Seconds 60 CMTraceLog -Message "Waiting 1 Minutes for CMClient to become active" -Type 1 -LogFile $LogFile Start-Sleep -Seconds 60 Disable-ProvMode Reset-TaskSequence #Set OSRollbackRan key to 2, to know where in the script it was if a reboot should occur Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "2" -Force } if ((Get-ItemPropertyValue -Path "$RegistryTemp" -Name "OSRollbackRan") -eq "2") { CMTraceLog -Message "Triggering CM Hardware Inventory" -Type 1 -LogFile $LogFile Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000001}" CMTraceLog -Message "Triggering CM Machine Policy Retrieval Cycle" -Type 1 -LogFile $LogFile Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000021}" CMTraceLog -Message "Triggering CM Machine Policy Evaluation Cycle" -Type 1 -LogFile $LogFile Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000022}" CMTraceLog -Message "Waiting 1 Minutes for Policy to Update" -Type 1 -LogFile $LogFile Start-Sleep -Seconds 60 CMTraceLog -Message "Triggering CM Machine Policy Evaluation Cycle" -Type 1 -LogFile $LogFile Invoke-WMIMethod -Namespace root\ccm -Class SMS_CLIENT -Name TriggerSchedule "{00000000-0000-0000-0000-000000000113}" #Added Phase of Failure to inventoried Values #Set-ItemProperty -Path "$RegistryPathFull" -Name "RollBackPhase" -Value "$(Get-ItemPropertyValue -Path $RegistryRollBack -Name "Phase")" -Force #Set-ItemProperty -Path "$RegistryPathFull" -Name "WaaS_Stage" -Value "Deployment_RollBack" -Force #Set OSRollbackRan key to 3, to know where in the script it was if a reboot should occur, 3 basically means every time the script is triggered, it will do nothing, because it's already completed the required steps Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "3" -Force } if ((Get-ItemPropertyValue -Path "$RegistryTemp" -Name "OSRollbackRan") -eq "3") { if ((Get-ItemPropertyValue -Path $RegistryRollBack -Name "Phase") -ne "5") { #Grab Logs and Backup To Server CMTraceLog -Message "Backing Up Logs to Server" -Type 1 -LogFile $LogFile CMTraceLog -Message "Location: $TargetRoot\$LogID" -Type 1 -LogFile $LogFile #region Prepare Target write-verbose "Log Archive Tool 1.0.<Version>" write-verbose "Create Target $TargetRoot\$LogID" new-item -itemtype Directory -Path $TargetRoot\$LogID -force -erroraction SilentlyContinue | out-null $TagFile = "$TargetRoot\$LogID\$($LogID.Replace('\','_'))" #endregion #region Create temporary Store $TempPath = [System.IO.Path]::GetTempFileName() remove-item $TempPath new-item -type directory -path $TempPath -force | out-null foreach ( $Item in $Path ) { $TmpTarget = (join-path $TempPath ( split-path -NoQualifier $Item )) write-Verbose "COPy $Item to $TmpTarget" copy-item -path $Item -Destination $TmpTarget -Force -Recurse -exclude $Exclude -ErrorAction SilentlyContinue } Compress-Archive -path "$TempPath\*" -DestinationPath "$TargetRoot\$LogID\$($LogID.Replace('\','_'))-$([datetime]::now.Tostring('s').Replace(':','-')).zip" -Force remove-item $tempPath -Recurse -Force #endregion CMTraceLog -Message "Finished Backing Up Logs to Server" -Type 1 -LogFile $LogFile } #Set-ItemProperty -Path "$RegistryPathFull" -Name "RollBackLog" -Value "LogLocation: $TargetRoot\$LogID" -Force Set-ItemProperty -Path "$RegistryTemp" -Name "OSRollbackRan" -Value "4" -Force CMTraceLog -Message "---Exiting $ScriptName Script---" -Type 1 -LogFile $LogFile Start-Process "C:\Windows\ccm\CcmEval.exe" Exit } } else {if ((Test-Path "$RegistryTemp") -eq 'True'){Remove-Item -Path "$RegistryTemp" -Force}}
Download (if Copy / Paste isn’t your thing) : Rollback Recovery Stand Alone (1534 downloads )
Posted on GARYTOWN.COM