Copying sheets while preserving digital signatures

Question

-- Edit: this is now part of the bigger question of how to reliably move sheets about in this question's context --

(Note: during preparing this post and testing solutions, I probably have already answered my own question. Just posting this in the hope anyone smarter than me can come up with something. Anyway, it's still a good resource for future searchers I guess.)

Problem description

I made an Excel solution for one of my customers which has tons of VBA in it. I therefore naturally signed the VBA code, so my customer doesn't get the macro security messages. However, one thing this solution does is making copies of a template sheet in the same workbook. The template sheet is found on it's code name, and all copies of the sheet are from then on recognized by their code name being derived from this (having a trailing sequence nr.) - they need to be identified and handled later on again.

Quite innocent on first sight, but when I demoed the solution and tried to save it I instantly got:

"You have modified a signed project. You do not have the correct key to sign this project. The signature will be discarded."

after which the signature was discarded, and on re-open the macro security prompts put themselves to good use. Not a good impression :(

The code goes like this in simplified form:

1. There is a (hidden) "template" sheet in the workbook that acts as the source for new sheets (it has no VBA code behind it nor any ActiveX or form controls on it);
2. A ribbon button calls VBA code that uses Worksheet.Copy to make a copy of this sheet (and modifies the copy, but that is irrelevant here);
3. On next save, Excel wants to discard the digital signature.

When I perform the same actions manually on a machine that doesn't have my certificate, I get the same experience. (A lesson: always test on truly blank systems before demoing anything...)

Possible cause

I've searched on this a bit (see e.g. ozgrid.com and answers.microsoft.com), and while remarkedly few people run into this, it seems like a sort-of inevitable thing. The reason behind it I suspect goes like this:

1. Although the template sheet has no 'real' VBA code on it, the VBA module does exist and has some not-insignificant content;
2. Copying this sheet creates a new sheet with a thus seemingly 'empty' but still existing and thus significant VBA module;
3. The hash of the 'total' VBA project is thus altered and the signature is lost.

According to the post on ozgrid.com, this also happens on deletion of sheets, which is explained by the above. It also suggests creating new sheets without the VBA IDE open doesn't trigger this, and deleting these new sheets works too. But once you go to the VBA IDE, all sheets currently present become 'non-deletable' again.

I suspect that when you add a new worksheet without the VBA editor open, Excel adds a worksheet with truly no VBA module added to it, so the project hash will not update. These sheets thus can also be deleted for the same reason. Opening the VBA editor in turn makes the IDE query for the modules in the workbook, at which time these still missing modules get created, baking their presence into the hash, which in turn also makes them uncopyable because their VBA footprint has become non-zero.

Solutions

Now the $1,000,000 question is: how can we work around this? There's some smart people on this site, so maybe we can come up with an out-of-the-box solution?

A useage detail that will make this easier (at least for me): the customer is the only one adding sheets, and he is never going to enter the IDE. It would be nice if I couldn't inadvertently mess up a build just by forgetfully entering the IDE, though.

I've already tried several possible solutions, creating them on a computer with my signature, and testing them on a computer without my signature. For now I'm using Excel 2010 32-bit exclusively for these tests, as that's either all I have, and it is also the version me and my customer are most interested in.

Non-solution 1

Delete all VBA code from the template sheet via the IDE, so it has no contribution to the hash.

If only it were so simple... This didn't work, so probably the existence of the module itself and/or it's meta-data (like it's name) is also hashed, which doesn't sound unreasonable. Or you simply cannot remove all VBA code since the IDE has the tendency to always append an empty line (so a single CrLf is as empty as you can make it this way, though it's CodeModule.CountOfLines return 0 on it). Or the entire VBA code module's content is retrieved and hashed, such that the terminating NULL char or leading 0 byte count contributes to the hash. Anyway, no luck here.

As a test I added a macro that tells which VBA modules there are, and how many lines they contain. Using this, a direct copy of the 'emptied' template sheet still has 0 lines but the signature is lost, while a newly inserted sheet shows up in the VBModules collection and even has 2 lines (the default Option Explicit) and the signature sticks nontheless on save...

But Excel might just be outsmarting us, with that 2-lined Option Explicit being a virtual one, or even the presence of the VBA module in the first place being virtual. When I made the macro also list all sheets with their code names, it turns out these 'safe' sheets have an empty code name (0-length string), indeed indicating they have no code module at all.

Non-solution 2

Create a fresh new sheet instead, and only copy over the contents of the template sheet.

While this does work, it seems a bit iffy to me; I do not believe a mere sourceSheet.Cells.Copy destSheet.Cells will copy absolutely everything the user can throw on it... I'd rather thus keep using the build-in Worksheet.Copy function to be safe and to not have to write piles of special code for every conceivable detail.

As a case on point: sourceSheet.Cells.Copy destSheet.Cells e.g. does copy over worksheet-specific named ranges, but apparently only if they're actually used on the sheet itself. Unreferenced names just vanish in the copy! Talk about special-case copy code I'd have to write...

And then there's the copied sheet not getting any code name assigned at all, which I currently need to recognize them.

Non-solution 3

Create a new temporary workbook, Worksheet.Copy the sheet to there, note it's name, explicitly save it as an .xlsx file to get rid of any VBA module, close and re-open the temp workbook to get rid of any old in-memory cruft, find it again by name, then Worksheet.Move it back to the source workbook.

This works! Without the actual workbook re-open it doesn't, so I guess the in-memory representation just cannot be 'scrubbed' easily enough to not do any harm.

However... The new sheet again doesn't get a code name at all, and even more: I do not like this sheet moving around to unrelated workbooks; while in a quick test any references to other sheets in the original workbook were conserved (and not even got expanded to include the workbook name or path!), I am still a bit uneasy about this... Who knows what type of content users might throw at it...

<Paranoid mode="on">And who knows what type of confidential information will be in there, which I do not want to have the responsibility for when it ends up leaking from the Temp folder without their knowing.</Paranoid>

Non-solution 4

Create a new, empty, temporary sheet as well as a Worksheet.Copy of the template, then replace the true copy's VBA module with the temporary sheet's one. Or just nuke the VBA module as a whole.

I just can't devise a way to do this. VBA itself won't let you do it it seems, and then again I do not want my customers to have to turn on the 'Allow access to the VB project' option for this alone. And I suspect were I able to do this, the damage would already have been done before I could nuke the code module again.

Non-solution 5

Create a macro that is only visible to me (the developer), that creates a perfect copy of the template sheet via either solution 2 or 3, and discards the original template sheet, replacing it with the VBA-scrubbed copy. To be used by me as the last step just before delivering it to the customer.

Solution 2's caveats are less important here because I do know myself what's on the template sheet when I make a new version delivery, so the amount of code needed for a perfect copy is minimal and can be controlled. But then 3 just seems safer and easier... I'll have to pick one.

Since I access the template sheet on it's VBA code name by just using shtTemplate. directly instead of ThisWorkbook.Worksheets("Template")., that apparently complicates it all too much for Excel to switch it in-and-out on the fly. All my attempts so far either failed or just made Excel crash hard on me. No love there :(

I tried this again by manipulating a copy loaded in a second Excel set to msoAutomationSecurityForceDisable, thus avoiding a running VBA host being undermined, also saving and re-opening after almost every update. But that led nowhere either, giving errors like "Automation error - Catastrophic failure" when opening the scrubbed workbook, or mightily corrupting the new workbook (the ThisWorkbook module being duplicated for each sheet module in the project explorer with a derived name).

Maybe-solution 6

Re-write all VBA to not use the hard-coded template sheet's code name, but storing this name on a settings sheet, then applying solution 5 above.

The code finally works, not even having to use a second staging Excel; no crashes nor corruptions! But this code works only insofar that I cannot for the life of me get the code to give the scrubbed sheet a valid code name again; it remains a zero-length string. And no run-time errors to indicate this either. When I have the IDE open during this, the code name is set correctly though.

Which leads me to believe that having a code name on your sheet implies it having a non-null code module, which implies it messing with the digital signature. And that's... not so unexpected really, in hindsight.

Final solution

Which leads me to believe there is just no way whatsoever that I could create a template sheet that both:

1. Is safe to copy via Worksheet.Copy without losing the signature, and
2. Has no code module while having a non-null code name.

The only solution I see so far is thus to indeed use a scrubbed template sheet to be able to use Worksheet.Copy, but to find and identify it and it's resulting sheets by other means than by their code name. There is a user-hidden section on it that I might add a "This is the template/copy" status to, though it makes my inner perfectionist cringe.

However, if anyone feels like experimenting, it would be nice to have a few more alternatives! I can post code samples when needed.

Answer 1

It's a lot to take in, and I do not pretnd this will answer will solve all your problems. But I once wrote a function called SoftLink which would take up to 4 parameters (i) Boolean: CellRef (or NamedRange) (ii) String: Range (iii) String: WorksheetName (iv) String: WorkbookName which would break any link with any cells and then you resolve the string parameters in VBA code.

There no doubt a performance hit with this approach but it is one way to solve Link hell.

Example calling formulas

=softlink(FALSE,"Foo")
=softlink(TRUE,"C4","Sheet1","Book2")
=softlink(TRUE,"D5","Sheet2")

and I have knocked up from memory an implementation. I have a phobia of On Errors .... so forgive some strange loopings in the subroutines.

Option Explicit

Function SoftLink(ByVal bIsCell As Boolean, ByVal sRangeName As String, _
                    Optional sSheetName As String, Optional sBookName As String) As Variant

    Dim vRet As Variant
    If Len(sRangeName) = 0 Then vRet = "#Cannot resolve null range name!": GoTo SingleExit '* fast fail


    Dim rngCaller As Excel.Range
    Set rngCaller = Application.Caller

    Dim wsCaller As Excel.Worksheet
    Set wsCaller = rngCaller.Parent

    Dim wbCaller As Excel.Workbook
    Set wbCaller = wsCaller.Parent

    Dim wb As Excel.Workbook

    If Len(sBookName) > 0 Then
        vRet = FindWorkbookWithoutOnErrorResumeNext(sBookName, wb)
        If Len(vRet) > 0 Then GoTo ErrorMessageExit
    Else
        Set wb = wbCaller
    End If
    Debug.Assert Not wb Is Nothing
    Dim ws As Excel.Worksheet
    If Len(sSheetName) > 0 Then
        vRet = FindWorksheetWithoutOnErrorResumeNext(wb, sSheetName, ws)
        If Len(vRet) > 0 Then GoTo ErrorMessageExit

    Else
        Set ws = wsCaller
    End If

    Dim rng As Excel.Range
    If bIsCell Then
        vRet = AcquireCellRange(ws, sRangeName, rng)
        If Len(vRet) > 0 Then GoTo ErrorMessageExit
    Else
        vRet = AcquireNamedRangeWithoutOERN(ws, sRangeName, rng)
        If Len(vRet) > 0 Then GoTo ErrorMessageExit
    End If

    SoftLink = rng.Value2
SingleExit:
    Exit Function
ErrorMessageExit:
    SoftLink = vRet
    GoTo SingleExit
End Function

Function AcquireCellRange(ByVal ws As Excel.Worksheet, ByVal sRangeName As String, ByRef prng As Excel.Range) As String

    On Error GoTo FailedCellRef
    Set prng = ws.Range(sRangeName)

SingleExit:
    Exit Function
FailedCellRef:
    AcquireCellRange = "#Could not resolve range name '" & sRangeName & "' on worksheet name '" & ws.Name & "' in workbook '" & ws.Parent.Name & "'!"

End Function


Function AcquireNamedRangeWithoutOERN(ByVal ws As Excel.Worksheet, ByVal sRangeName As String, ByRef prng As Excel.Range) As String

    '* because I do not like OERN
    Dim oNames As Excel.Names

    Dim bSheetScope As Long
    For bSheetScope = True To False

        Set oNames = VBA.IIf(bSheetScope, ws.Names, ws.Parent.Names)

        Dim namLoop As Excel.Name
        For Each namLoop In oNames
            If VBA.StrComp(namLoop.Name, sRangeName, vbTextCompare) = 0 Then

                Set prng = ws.Range(sRangeName)
                GoTo SingleExit
            End If

        Next
    Next

ErrorMessageExit:
    AcquireNamedRangeWithoutOERN = "#Could not resolve range name '" & sRangeName & "' on worksheet name '" & ws.Name & "' in workbook '" & ws.Parent.Name & "'!"
SingleExit:
    Exit Function

End Function

Function FindWorksheetWithoutOnErrorResumeNext(ByVal wb As Excel.Workbook, ByVal sSheetName As String, ByRef pws As Excel.Worksheet) As String
    '* because I do not like OERN
    Dim wsLoop As Excel.Worksheet
    For Each wsLoop In wb.Worksheets
        If VBA.StrComp(wsLoop.Name, sSheetName, vbTextCompare) = 0 Then
            Set pws = wsLoop

            GoTo SingleExit
        End If

    Next wsLoop
ErrorMessageExit:
    FindWorksheetWithoutOnErrorResumeNext = "#Could not resolve worksheet name '" & sSheetName & "' in workbook '" & wb.Name & "'!"
SingleExit:
    Exit Function
End Function


Function FindWorkbookWithoutOnErrorResumeNext(ByVal sBookName As String, ByRef pwb As Excel.Workbook) As String
    '* because I do not like OERN
    Dim wbLoop As Excel.Workbook
    For Each wbLoop In Application.Workbooks
        If VBA.StrComp(wbLoop.Name, sBookName, vbTextCompare) = 0 Then
            Set pwb = wbLoop

            GoTo SingleExit
        End If

    Next wbLoop
ErrorMessageExit:
    FindWorkbookWithoutOnErrorResumeNext = "#Could not resolve workbook name '" & sBookName & "'!"
SingleExit:
    Exit Function
End Function