make the handling of import paths more robust
First, make isPrivate panic on malformed import paths, since that should
never happen. This catches the errors that some users had run into with
packages like gopkg.in/yaml.v2 and github.com/satori/go.uuid:
panic: malformed import path "gopkg.in/garbletest%2ev2": invalid char '%'
This seems to trigger when a module path contains a dot after the first
element, *and* that module is fetched via the proxy. This results in the
toolchain URL-encoding the second dot, and garble ends up seeing that
encoded path.
We reproduce this behavior with a fake gopkg.in module added to the test
module proxy. Using yaml.v2 directly would have been easier, but it's
pretty large. Note that we tried a replace directive, but that does not
trigger the URL-encoding bug.
Also note that we do not obfuscate the gopkg.in package; that's fine, as
the isPrivate path validity check catches the bug either way.
For now, make initImport use url.PathUnescape to work around this issue.
The underlying bug is likely in either the goobj2 fork, or in the
upstream Go toolchain itself.
hashImport also gives a better error if it cannot find a package now,
rather than just an "empty seed" panic.
Finally, the sanity check in isPrivate unearthed the fact that we do not
support garbling test packages at all, since they were invalid paths
which never matched GOPRIVATE. Add an explicit check and TODO about
that.
Fixes #224.
Fixes #228.
4 years ago
|
|
|
env GOPRIVATE='test/main,private.source'
|
|
|
|
|
rewrite go:linkname directives with garbled names (#200)
If code includes a linkname directive pointing at a name in an imported
package, like:
//go:linkname localName importedpackage.RemoteName
func localName()
We should rewrite the comment to replace "RemoteName" with its
obfuscated counterpart, if the package in question was obfuscated and
that name was as well.
We already had some code to handle linkname directives, but only to
ensure that "localName" was never obfuscated. This behavior is kept, to
ensure that the directive applies to the right name. In the future, we
could instead rewrite "localName" in the directive, like we do with
"RemoteName".
Add plenty of tests, too. The linkname directive used to be tested in
imports.txt and syntax.txt, but that was hard to maintain as each file
tested different edge cases.
Now that we have build caching, adding one extra testscript file isn't a
big problem anymoree. Add linkname.txt, which is self-explanatory. The
other two scripts also get a bit less complex.
Fixes #197.
4 years ago
|
|
|
garble build
|
|
|
|
exec ./main$exe
|
|
|
|
cmp stderr main.stderr
|
|
|
|
|
rewrite go:linkname directives with garbled names (#200)
If code includes a linkname directive pointing at a name in an imported
package, like:
//go:linkname localName importedpackage.RemoteName
func localName()
We should rewrite the comment to replace "RemoteName" with its
obfuscated counterpart, if the package in question was obfuscated and
that name was as well.
We already had some code to handle linkname directives, but only to
ensure that "localName" was never obfuscated. This behavior is kept, to
ensure that the directive applies to the right name. In the future, we
could instead rewrite "localName" in the directive, like we do with
"RemoteName".
Add plenty of tests, too. The linkname directive used to be tested in
imports.txt and syntax.txt, but that was hard to maintain as each file
tested different edge cases.
Now that we have build caching, adding one extra testscript file isn't a
big problem anymoree. Add linkname.txt, which is self-explanatory. The
other two scripts also get a bit less complex.
Fixes #197.
4 years ago
|
|
|
! binsubstr main$exe 'localName' 'globalConst' 'globalVar' 'globalType' 'valuable information' 'private.source' 'remoteIntReturn' 'intReturn' 'neverInlined'
|
|
|
|
|
|
|
|
[short] stop # no need to verify this with -short
|
|
|
|
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
# Check that the program works as expected without garble.
|
rewrite go:linkname directives with garbled names (#200)
If code includes a linkname directive pointing at a name in an imported
package, like:
//go:linkname localName importedpackage.RemoteName
func localName()
We should rewrite the comment to replace "RemoteName" with its
obfuscated counterpart, if the package in question was obfuscated and
that name was as well.
We already had some code to handle linkname directives, but only to
ensure that "localName" was never obfuscated. This behavior is kept, to
ensure that the directive applies to the right name. In the future, we
could instead rewrite "localName" in the directive, like we do with
"RemoteName".
Add plenty of tests, too. The linkname directive used to be tested in
imports.txt and syntax.txt, but that was hard to maintain as each file
tested different edge cases.
Now that we have build caching, adding one extra testscript file isn't a
big problem anymoree. Add linkname.txt, which is self-explanatory. The
other two scripts also get a bit less complex.
Fixes #197.
4 years ago
|
|
|
go build
|
|
|
|
exec ./main$exe
|
|
|
|
cmp stderr main.stderr
|
|
|
|
|
|
|
|
binsubstr main$exe 'globalVar' # 'globalType' only matches on go < 1.15
|
|
|
|
! binsubstr main$exe 'localName' 'globalConst' 'remoteIntReturn' 'intReturn'
|
|
|
|
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
-- extra/go.mod --
|
make the handling of import paths more robust
First, make isPrivate panic on malformed import paths, since that should
never happen. This catches the errors that some users had run into with
packages like gopkg.in/yaml.v2 and github.com/satori/go.uuid:
panic: malformed import path "gopkg.in/garbletest%2ev2": invalid char '%'
This seems to trigger when a module path contains a dot after the first
element, *and* that module is fetched via the proxy. This results in the
toolchain URL-encoding the second dot, and garble ends up seeing that
encoded path.
We reproduce this behavior with a fake gopkg.in module added to the test
module proxy. Using yaml.v2 directly would have been easier, but it's
pretty large. Note that we tried a replace directive, but that does not
trigger the URL-encoding bug.
Also note that we do not obfuscate the gopkg.in package; that's fine, as
the isPrivate path validity check catches the bug either way.
For now, make initImport use url.PathUnescape to work around this issue.
The underlying bug is likely in either the goobj2 fork, or in the
upstream Go toolchain itself.
hashImport also gives a better error if it cannot find a package now,
rather than just an "empty seed" panic.
Finally, the sanity check in isPrivate unearthed the fact that we do not
support garbling test packages at all, since they were invalid paths
which never matched GOPRIVATE. Add an explicit check and TODO about
that.
Fixes #224.
Fixes #228.
4 years ago
|
|
|
module private.source/extra
|
|
|
|
|
|
|
|
go 1.15
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
-- extra/extra.go --
|
|
|
|
package extra
|
|
|
|
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
func Func() string {
|
|
|
|
return "This is a separate module to obfuscate."
|
|
|
|
}
|
|
|
|
-- go.mod --
|
|
|
|
module test/main
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
|
|
|
|
go 1.15
|
|
|
|
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
// We include an extra module to obfuscate, included in the same original source
|
|
|
|
// code via a replace directive.
|
|
|
|
require private.source/extra v0.0.0-00010101000000-000000000000
|
|
|
|
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
replace private.source/extra => ./extra
|
|
|
|
-- main.go --
|
|
|
|
package main
|
|
|
|
|
|
|
|
import (
|
|
|
|
"encoding/json"
|
|
|
|
"go/ast"
|
|
|
|
"reflect"
|
|
|
|
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
"private.source/extra"
|
|
|
|
"test/main/sub"
|
|
|
|
)
|
|
|
|
|
|
|
|
// This comment contains valuable information. Ensure it's not in the final binary.
|
|
|
|
var V interface{}
|
|
|
|
|
|
|
|
type T struct {
|
|
|
|
ast.Node
|
|
|
|
*ast.Ident
|
|
|
|
}
|
|
|
|
|
|
|
|
type EncodingT struct {
|
|
|
|
Foo int
|
|
|
|
}
|
|
|
|
|
|
|
|
type Embedded int
|
|
|
|
|
|
|
|
type Embedding struct {
|
|
|
|
Embedded
|
|
|
|
}
|
|
|
|
|
|
|
|
type embedded int
|
|
|
|
|
|
|
|
type embedding struct {
|
|
|
|
embedded
|
|
|
|
}
|
|
|
|
|
|
|
|
// embedded fields whose type is in the universe scope used to crash garble
|
|
|
|
type EmbeddingUniverseScope struct {
|
|
|
|
error
|
|
|
|
int
|
|
|
|
string
|
|
|
|
}
|
|
|
|
|
rewrite go:linkname directives with garbled names (#200)
If code includes a linkname directive pointing at a name in an imported
package, like:
//go:linkname localName importedpackage.RemoteName
func localName()
We should rewrite the comment to replace "RemoteName" with its
obfuscated counterpart, if the package in question was obfuscated and
that name was as well.
We already had some code to handle linkname directives, but only to
ensure that "localName" was never obfuscated. This behavior is kept, to
ensure that the directive applies to the right name. In the future, we
could instead rewrite "localName" in the directive, like we do with
"RemoteName".
Add plenty of tests, too. The linkname directive used to be tested in
imports.txt and syntax.txt, but that was hard to maintain as each file
tested different edge cases.
Now that we have build caching, adding one extra testscript file isn't a
big problem anymoree. Add linkname.txt, which is self-explanatory. The
other two scripts also get a bit less complex.
Fixes #197.
4 years ago
|
|
|
// TODO: test that go:noinline still works without using debugdir
|
|
|
|
|
|
|
|
//go:noinline
|
|
|
|
func neverInlined() {
|
|
|
|
println("This func is never inlined.")
|
|
|
|
}
|
|
|
|
|
|
|
|
func main() {
|
|
|
|
switch V := V.(type) {
|
|
|
|
case int:
|
|
|
|
var _ int = V
|
|
|
|
case nil:
|
|
|
|
println("nil case")
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ = reflect.TypeOf(EncodingT{})
|
|
|
|
enc, _ := json.Marshal(EncodingT{Foo: 3})
|
|
|
|
println(string(enc))
|
|
|
|
scopesTest()
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
println(extra.Func())
|
|
|
|
sub.Test()
|
rewrite go:linkname directives with garbled names (#200)
If code includes a linkname directive pointing at a name in an imported
package, like:
//go:linkname localName importedpackage.RemoteName
func localName()
We should rewrite the comment to replace "RemoteName" with its
obfuscated counterpart, if the package in question was obfuscated and
that name was as well.
We already had some code to handle linkname directives, but only to
ensure that "localName" was never obfuscated. This behavior is kept, to
ensure that the directive applies to the right name. In the future, we
could instead rewrite "localName" in the directive, like we do with
"RemoteName".
Add plenty of tests, too. The linkname directive used to be tested in
imports.txt and syntax.txt, but that was hard to maintain as each file
tested different edge cases.
Now that we have build caching, adding one extra testscript file isn't a
big problem anymoree. Add linkname.txt, which is self-explanatory. The
other two scripts also get a bit less complex.
Fixes #197.
4 years ago
|
|
|
neverInlined()
|
|
|
|
}
|
|
|
|
|
|
|
|
-- scopes.go --
|
|
|
|
package main
|
|
|
|
|
|
|
|
const globalConst = 1
|
|
|
|
|
|
|
|
type globalType int
|
|
|
|
|
|
|
|
var (
|
|
|
|
globalVar = 1
|
|
|
|
globalVarTyped globalType = 1
|
|
|
|
)
|
|
|
|
|
|
|
|
func scopesTest() {
|
|
|
|
println(globalVar, globalConst, globalVarTyped)
|
|
|
|
const localNameConst = 1
|
|
|
|
|
|
|
|
localNameShort := 4
|
|
|
|
|
|
|
|
type localNameType int
|
|
|
|
|
|
|
|
var (
|
|
|
|
localNameVar = 5
|
|
|
|
localNameTypeVar localNameType = 1
|
|
|
|
)
|
|
|
|
|
|
|
|
println(localNameConst, localNameShort, localNameVar, localNameTypeVar, input("input"))
|
|
|
|
}
|
|
|
|
|
|
|
|
func input(localNameParam string) (localNameReturn string) { return localNameParam }
|
|
|
|
|
|
|
|
-- sub/names.go --
|
|
|
|
package sub
|
|
|
|
|
|
|
|
var someGlobalVar0 = "0"
|
|
|
|
var someGlobalVar1 = "1"
|
|
|
|
var someGlobalVar2 = "2"
|
|
|
|
|
|
|
|
func Test() {
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
var A, B, C, D, E string
|
|
|
|
noop(A, B, C, D, E)
|
|
|
|
if someGlobalVar0 != "0" || someGlobalVar1 != "1" || someGlobalVar2 != "2"{
|
|
|
|
panic("name collision detected")
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func noop(...interface{}) {}
|
|
|
|
-- main.stderr --
|
|
|
|
nil case
|
|
|
|
{"Foo":3}
|
|
|
|
1 1 1
|
|
|
|
1 4 5 1 input
|
testdata: make syntax.txt pass when offline
The test intended to use an extra module to be obfuscated, rsc.io/quote,
which we were bundling in the local proxy as well. Unfortunately, the
use of GOPRIVATE also meant that we did not actually fetch the module
from the proxy, and we would instead do a full roundtrip to the internet
to "git clone" the actual upstream repository.
To prevent that roundtrip, instead use a locally replaced module. This
fits the syntax.txt test too, since it's one more edge case that we want
to make sure works well with garble. Since rsc.io/quote is used in
another test, simply make up our own tiny module.
Reduces a 'go test -run Syntax/syntax' run with warm cache from ~5s to
~0.5s, thanks to removing the multiple roundtrips. A warm 'go test' run
still sits at ~6s, since we still need that much CPU time in total.
While at it, fix a staticcheck warning and fix inconsistent indentation
in a couple of tests.
5 years ago
|
|
|
This is a separate module to obfuscate.
|
rewrite go:linkname directives with garbled names (#200)
If code includes a linkname directive pointing at a name in an imported
package, like:
//go:linkname localName importedpackage.RemoteName
func localName()
We should rewrite the comment to replace "RemoteName" with its
obfuscated counterpart, if the package in question was obfuscated and
that name was as well.
We already had some code to handle linkname directives, but only to
ensure that "localName" was never obfuscated. This behavior is kept, to
ensure that the directive applies to the right name. In the future, we
could instead rewrite "localName" in the directive, like we do with
"RemoteName".
Add plenty of tests, too. The linkname directive used to be tested in
imports.txt and syntax.txt, but that was hard to maintain as each file
tested different edge cases.
Now that we have build caching, adding one extra testscript file isn't a
big problem anymoree. Add linkname.txt, which is self-explanatory. The
other two scripts also get a bit less complex.
Fixes #197.
4 years ago
|
|
|
This func is never inlined.
|