Well I figured out why I was having so much trouble.
I started removing files from the mpq so I was dealing with fewer variables. I found I could get the correct value for all the isolated files- except war3map.j. Even the empty war3map.j file gives a non-zero key.
I think it's prepending common.j or something else onto war3map.j before running the check.
Code:
Public Function loopcheck(ByVal s As IO.Stream, ByVal valwo As Integer, ByVal valbo As Integer, ByVal wordo As Integer, ByVal byteo As Integer, Optional ByVal length As Integer = -1) As UInteger
If length = -1 Then length = CInt(s.Length)
Dim val As UInteger = 0
Dim r As New IO.BinaryReader(s)
For repeat As Integer = 1 To length - 3 Step 4
val = ShiftRotateLeft(val, valwo) Xor ShiftRotateLeft(r.ReadUInt32(), wordo)
Next repeat
For repeat As Integer = 1 To length Mod 4
val = ShiftRotateLeft(val, valbo) Xor ShiftRotateLeft(CUInt(r.ReadByte()), byteo)
Next repeat
Return val
End Function
Public Function loopCheck2(ByVal m As MPQ.Archive) As UInteger
Dim val As UInteger = 0
Dim offset As Integer = 3
Dim ss() As String = {"war3map.j", "war3map.doo", "war3map.wpm", "war3map.w3e"}
For Each s As String In ss
If m.hashTable.contains(s) Then
offset += 3
val = val Xor loopcheck(New MPQ.FileStream(m, s), 3, 3, offset, offset)
If s = "war3map.j" Then val = val Xor CUInt(973126876)
End If
Next s
Return val
End Function 
