UPDATE 3/6 - I've updated to Insider Build 15048, and this no longer works to remove that annoying icon. Hopefully once Creator Edition is finalized, they will have this ability again to remove that icon either via Registry or GPO.
I clicked on it, and it opened Edge to a default performance Page. Just seems like more advertising to me. I was hoping it would open the current page I had active in Edge. (Maybe they make it useful in RTM??, one can hope)
While changing these settings did nothing, I took the info here and create a new setting:
Once I created that setting and set it to “1”, IE removed the button.
Worked instantly, closed and opened IE, button Gone!
For OSD, just add that line into a “Run Command Line” step
Hope you find this helpful