code/avo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tests/thirdparty: multiple packages per project (#262)

Browse Source
This commit is contained in:
Michael McLoughlin
2022-04-24 20:20:11 -07:00
committed by GitHub
parent 96c4d85995
commit 84071ae4f9
14 changed files with 1051 additions and 894 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/thirdparty.yml vendored
View File

@@ -29,4 +29,4 @@ jobs:
persist-credentials: false
- name: Run Third-Party Tests
working-directory: tests/thirdparty
run: go test -net -pkgs packages.json
run: go test -net -prjs projects.json

16
README.md
View File

@@ -193,23 +193,23 @@ Implementations of full algorithms:
Popular projects using `avo`:
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fgolang.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [golang / **go**](https://github.com/golang/go)
:star: 98269
:star: 98315
> The Go programming language
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fklauspost.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [klauspost / **compress**](https://github.com/klauspost/compress) / **s2**
:star: 2703
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fklauspost.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [klauspost / **compress**](https://github.com/klauspost/compress)
:star: 2706
> Optimized Go Compression Packages
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fgolang.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [golang / **crypto**](https://github.com/golang/crypto) / **curve25519**
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fgolang.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [golang / **crypto**](https://github.com/golang/crypto)
:star: 2402
> [mirror] Go supplementary cryptography libraries
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fklauspost.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [klauspost / **reedsolomon**](https://github.com/klauspost/reedsolomon)
:star: 1437
:star: 1438
> Reed-Solomon Erasure Coding in Go
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fsegmentio.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [segmentio / **asm**](https://github.com/segmentio/asm)
:star: 685
:star: 686
> Go library providing algorithms optimized to leverage the characteristics of modern CPUs
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fzeebo.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [zeebo / **blake3**](https://github.com/zeebo/blake3)
@@ -217,7 +217,7 @@ Popular projects using `avo`:
> Pure Go implementation of BLAKE3 with AVX2 and SSE4.1 acceleration
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Flukechampine.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [lukechampine / **blake3**](https://github.com/lukechampine/blake3)
:star: 278
:star: 277
> A pure-Go implementation of the BLAKE3 cryptographic hash function
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fzeebo.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [zeebo / **xxh3**](https://github.com/zeebo/xxh3)
@@ -229,7 +229,7 @@ Popular projects using `avo`:
> Bloom-filter based search index
<img src="https://images.weserv.nl?fit=cover&h=24&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fminio.png&w=24" width="24" height="24" hspace="4" valign="middle" /> [minio / **md5-simd**](https://github.com/minio/md5-simd)
:star: 108
:star: 109
> Accelerate aggregated MD5 hashing performance up to 8x for AVX512 and 4x for AVX2. Useful for server applications that need to compute many MD5 sums in parallel.
See the [full list of projects using `avo`](doc/adopters.md).

12
doc/adopters.md
View File

@@ -4,11 +4,11 @@
The Go programming language
<img src="https://images.weserv.nl?fit=cover&h=28&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fklauspost.png&w=28" width="28" height="28" hspace="4" valign="middle" /> [klauspost / **compress**](https://github.com/klauspost/compress) / **s2**
<img src="https://images.weserv.nl?fit=cover&h=28&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fklauspost.png&w=28" width="28" height="28" hspace="4" valign="middle" /> [klauspost / **compress**](https://github.com/klauspost/compress)
Optimized Go Compression Packages
<img src="https://images.weserv.nl?fit=cover&h=28&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fgolang.png&w=28" width="28" height="28" hspace="4" valign="middle" /> [golang / **crypto**](https://github.com/golang/crypto) / **curve25519**
<img src="https://images.weserv.nl?fit=cover&h=28&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fgolang.png&w=28" width="28" height="28" hspace="4" valign="middle" /> [golang / **crypto**](https://github.com/golang/crypto)
[mirror] Go supplementary cryptography libraries
@@ -48,7 +48,7 @@ Go implementation of the BLS12-381 pairing
filippo.io/edwards25519 — A safer, faster, and more powerful low-level edwards25519 Go implementation.
<img src="https://images.weserv.nl?fit=cover&h=28&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Flukechampine.png&w=28" width="28" height="28" hspace="4" valign="middle" /> [lukechampine / **us**](https://github.com/lukechampine/us) / **merkle/blake2b**
<img src="https://images.weserv.nl?fit=cover&h=28&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Flukechampine.png&w=28" width="28" height="28" hspace="4" valign="middle" /> [lukechampine / **us**](https://github.com/lukechampine/us)
An alternative interface to Sia
@@ -84,10 +84,6 @@ SPECK cipher
go-chaskey: an implementation of chaskey, an efficient MAC for microcontrollers
<img src="https://images.weserv.nl?fit=cover&h=28&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fericlagergren.png&w=28" width="28" height="28" hspace="4" valign="middle" /> [ericlagergren / **lwcrypto**](https://github.com/ericlagergren/lwcrypto) / **ascon**
NIST Lightweight Cryptography finalists
<img src="https://images.weserv.nl?fit=cover&h=28&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fericlagergren.png&w=28" width="28" height="28" hspace="4" valign="middle" /> [ericlagergren / **lwcrypto**](https://github.com/ericlagergren/lwcrypto) / **grain**
<img src="https://images.weserv.nl?fit=cover&h=28&mask=circle&maxage=7d&url=https%3A%2F%2Fgithub.com%2Fericlagergren.png&w=28" width="28" height="28" hspace="4" valign="middle" /> [ericlagergren / **lwcrypto**](https://github.com/ericlagergren/lwcrypto)
NIST Lightweight Cryptography finalists

12
internal/cmd/docgen/main.go
View File

@@ -32,7 +32,7 @@ var (
typ = flag.String("type", "", "documentation type")
tmpl = flag.String("tmpl", "", "explicit template file (overrides -type)")
output = flag.String("output", "", "path to output file (default stdout)")
pkgsfilename = flag.String("pkgs", "", "packages configuration")
prjsfilename = flag.String("prjs", "", "projects configuration")
)
func mainerr() (err error) {
@@ -57,19 +57,19 @@ func mainerr() (err error) {
return err
}
// Load third-party packages.
if *pkgsfilename == "" {
return errors.New("missing packages configuration")
// Load third-party projects.
if *prjsfilename == "" {
return errors.New("missing projects configuration")
}
pkgs, err := thirdparty.LoadPackagesFile(*pkgsfilename)
prjs, err := thirdparty.LoadProjectsFile(*prjsfilename)
if err != nil {
return err
}
// Execute.
data := map[string]interface{}{
"Packages": pkgs,
"Projects": prjs,
}
var buf bytes.Buffer

3
internal/cmd/docgen/templates/adopters.tmpl
View File

@@ -1,7 +1,6 @@
# Adopters
{{ range .Packages.Ranked }}
{{ range .Projects.Ranked }}
{{ avatar .Repository.Owner 28 }} [{{ .Repository.Owner }} / **{{ .Repository.Name }}**]({{ .Repository.URL }})
{{- if .SubPackage }} / **{{ .SubPackage}}**{{ end }}
{{ .Metadata.Description }}
{{ end -}}

3
internal/cmd/docgen/templates/readme.tmpl
View File

@@ -105,10 +105,9 @@ Implementations of full algorithms:
Popular projects using `avo`:
{{ range .Packages.Top 10 -}}
{{ range .Projects.Top 10 -}}
{{ if lt .Metadata.Stars 100 }}{{break}}{{ end -}}
{{ avatar .Repository.Owner 24 }} [{{ .Repository.Owner }} / **{{ .Repository.Name }}**]({{ .Repository.URL }})
{{- if .SubPackage }} / **{{ .SubPackage}}**{{ end }}
:star: {{ .Metadata.Stars }}
> {{ .Metadata.Description }}

6
script/doc
View File

@@ -3,9 +3,9 @@
# Generate some specific files with docgen.
go install ./internal/cmd/docgen
pkgs="tests/thirdparty/packages.json"
docgen -pkgs "${pkgs}" -type readme -output README.md
docgen -pkgs "${pkgs}" -type adopters -output doc/adopters.md
prjs="tests/thirdparty/projects.json"
docgen -prjs "${prjs}" -type readme -output README.md
docgen -prjs "${prjs}" -type adopters -output doc/adopters.md
# Process simple file embeddings with embedmd.
find . -name '*.md' | xargs embedmd -w

230
tests/thirdparty/config.go vendored
View File

@@ -1,4 +1,4 @@
// Package thirdparty executes integration tests based on third-party packages that use avo.
// Package thirdparty executes integration tests based on third-party projects that use avo.
package thirdparty
import (
@@ -45,6 +45,71 @@ type Metadata struct {
Stars int `json:"stars,omitempty"`
}
// Project defines an integration test based on a third-party project using avo.
type Project struct {
// Repository for the project. At the moment, all projects are available on
// github.
Repository GithubRepository `json:"repository"`
// Repository metadata.
Metadata Metadata `json:"metadata"`
// Default git branch. This is used when testing against the latest version.
DefaultBranch string `json:"default_branch,omitempty"`
// Version as a git sha, tag or branch.
Version string `json:"version"`
// If the project test has a known problem, record it by setting this to a
// non-zero avo issue number. If set, the project will be skipped in
// testing.
KnownIssue int `json:"known_issue,omitempty"`
// Packages within the project to test.
Packages []*Package `json:"packages"`
}
func (p *Project) defaults(set bool) {
for _, pkg := range p.Packages {
pkg.defaults(set)
}
}
// Validate project definition.
func (p *Project) Validate() error {
if p.DefaultBranch == "" {
return errors.New("missing default branch")
}
if p.Version == "" {
return errors.New("missing version")
}
if len(p.Packages) == 0 {
return errors.New("missing packages")
}
for _, pkg := range p.Packages {
if err := pkg.Validate(); err != nil {
return fmt.Errorf("package %s: %w", pkg.Name(), err)
}
}
return nil
}
// ID returns an identifier for the project.
func (p *Project) ID() string {
return strings.ReplaceAll(p.Repository.String(), "/", "-")
}
// Skip reports whether the project test should be skipped. If skipped, a known
// issue will be set.
func (p *Project) Skip() bool {
return p.KnownIssue != 0
}
// Reason returns the reason why the test is skipped.
func (p *Project) Reason() string {
return fmt.Sprintf("https://github.com/mmcloughlin/avo/issues/%d", p.KnownIssue)
}
// Step represents a set of commands to run as part of the testing plan for a
// third-party package.
type Step struct {
@@ -64,24 +129,11 @@ func (s *Step) Validate() error {
return nil
}
// Package defines an integration test based on a third-party package using avo.
// Package defines an integration test for a package within a project.
type Package struct {
// Repository the package belongs to. At the moment, all packages are
// available on github.
Repository GithubRepository `json:"repository"`
// Repository metadata.
Metadata Metadata `json:"metadata"`
// Default git branch. This is used when testing against the latest version.
DefaultBranch string `json:"default_branch,omitempty"`
// Version as a git sha, tag or branch.
Version string `json:"version"`
// Sub-package within the repository under test. All file path references
// will be relative to this directory. If empty the root of the repository
// is used.
// Sub-package within the project under test. All file path references will
// be relative to this directory. If empty the root of the repository is
// used.
SubPackage string `json:"pkg,omitempty"`
// Path to the module file for the avo generator package. This is necessary
@@ -95,32 +147,10 @@ type Package struct {
Setup []*Step `json:"setup,omitempty"`
// Steps to run the avo code generator.
Generate []*Step `json:"generate"` // generate commands to run
Generate []*Step `json:"generate"`
// Test steps. If empty, defaults to "go test ./...".
Test []*Step `json:"test,omitempty"`
// If the package test has a known problem, record it by setting this to a
// non-zero avo issue number. If set, the package will be skipped in
// testing.
KnownIssue int `json:"known_issue,omitempty"`
}
// ID returns an identifier for the package.
func (p *Package) ID() string {
pkgpath := path.Join(p.Repository.String(), p.SubPackage)
return strings.ReplaceAll(pkgpath, "/", "-")
}
// Skip reports whether the package test should be skipped. If skipped, a known
// issue will be set.
func (p *Package) Skip() bool {
return p.KnownIssue != 0
}
// Reason returns the reason why the test is skipped.
func (p *Package) Reason() string {
return fmt.Sprintf("https://github.com/mmcloughlin/avo/issues/%d", p.KnownIssue)
}
// defaults sets or removes default field values.
@@ -152,12 +182,6 @@ func applydefault(set bool, s, def string) string {
// Validate package definition.
func (p *Package) Validate() error {
if p.DefaultBranch == "" {
return errors.New("missing default branch")
}
if p.Version == "" {
return errors.New("missing version")
}
if p.Module == "" {
return errors.New("missing module")
}
@@ -181,6 +205,19 @@ func (p *Package) Validate() error {
return nil
}
// Name of the package.
func (p *Package) Name() string {
if p.IsRoot() {
return "root"
}
return p.SubPackage
}
// IsRoot reports whether the package is the root of the containing project.
func (p *Package) IsRoot() bool {
return p.SubPackage == ""
}
// Context specifies execution environment parameters for a third-party test.
type Context struct {
// Path to the avo version under test.
@@ -243,37 +280,72 @@ func (p *Package) Steps(c *Context) []*Step {
return steps
}
// Packages is a collection of third-party integration tests.
type Packages []*Package
// Test case for a given package within a project.
type Test struct {
Project *Project
Package *Package
}
func (p Packages) defaults(set bool) {
for _, pkg := range p {
pkg.defaults(set)
// ID returns an identifier for the test case.
func (t *Test) ID() string {
pkgpath := path.Join(t.Project.Repository.String(), t.Package.SubPackage)
return strings.ReplaceAll(pkgpath, "/", "-")
}
// Projects is a collection of third-party integration tests.
type Projects []*Project
func (p Projects) defaults(set bool) {
for _, prj := range p {
prj.defaults(set)
}
}
// Validate the package collection.
func (p Packages) Validate() error {
for _, pkg := range p {
if err := pkg.Validate(); err != nil {
return fmt.Errorf("package %s: %w", pkg.ID(), err)
// Validate the project collection.
func (p Projects) Validate() error {
seen := map[string]bool{}
for _, prj := range p {
// Project is valid.
if err := prj.Validate(); err != nil {
return fmt.Errorf("project %s: %w", prj.ID(), err)
}
// No duplicate entries.
id := prj.ID()
if seen[id] {
return fmt.Errorf("duplicate project %q", id)
}
seen[id] = true
}
return nil
}
// Ranked returns a copy of the packages list ranked in desending order of
// Tests returns all test cases for the projects collection.
func (p Projects) Tests() []*Test {
var ts []*Test
for _, prj := range p {
for _, pkg := range prj.Packages {
ts = append(ts, &Test{
Project: prj,
Package: pkg,
})
}
}
return ts
}
// Ranked returns a copy of the projects list ranked in desending order of
// popularity.
func (p Packages) Ranked() Packages {
ranked := append(Packages(nil), p...)
func (p Projects) Ranked() Projects {
ranked := append(Projects(nil), p...)
sort.SliceStable(ranked, func(i, j int) bool {
return ranked[i].Metadata.Stars > ranked[j].Metadata.Stars
})
return ranked
}
// Top returns the top n most popular packages.
func (p Packages) Top(n int) Packages {
// Top returns the top n most popular projects.
func (p Projects) Top(n int) Projects {
top := p.Ranked()
if len(top) > n {
top = top[:n]
@@ -281,44 +353,44 @@ func (p Packages) Top(n int) Packages {
return top
}
// LoadPackages loads a list of package configurations from JSON format.
func LoadPackages(r io.Reader) (Packages, error) {
var pkgs Packages
// LoadProjects loads a list of project configurations from JSON format.
func LoadProjects(r io.Reader) (Projects, error) {
var prjs Projects
d := json.NewDecoder(r)
d.DisallowUnknownFields()
if err := d.Decode(&pkgs); err != nil {
if err := d.Decode(&prjs); err != nil {
return nil, err
}
pkgs.defaults(true)
return pkgs, nil
prjs.defaults(true)
return prjs, nil
}
// LoadPackagesFile loads a list of package configurations from a JSON file.
func LoadPackagesFile(filename string) (Packages, error) {
// LoadProjectsFile loads a list of project configurations from a JSON file.
func LoadProjectsFile(filename string) (Projects, error) {
f, err := os.Open(filename)
if err != nil {
return nil, err
}
defer f.Close()
return LoadPackages(f)
return LoadProjects(f)
}
// StorePackages writes a list of package configurations in JSON format.
func StorePackages(w io.Writer, pkgs Packages) error {
// StoreProjects writes a list of project configurations in JSON format.
func StoreProjects(w io.Writer, prjs Projects) error {
e := json.NewEncoder(w)
e.SetIndent("", " ")
pkgs.defaults(false)
err := e.Encode(pkgs)
pkgs.defaults(true)
prjs.defaults(false)
err := e.Encode(prjs)
prjs.defaults(true)
return err
}
// StorePackagesFile writes a list of package configurations to a JSON file.
func StorePackagesFile(filename string, pkgs Packages) error {
// StoreProjectsFile writes a list of project configurations to a JSON file.
func StoreProjectsFile(filename string, prjs Projects) error {
f, err := os.Create(filename)
if err != nil {
return err
}
defer f.Close()
return StorePackages(f, pkgs)
return StoreProjects(f, prjs)
}

91
tests/thirdparty/config_test.go vendored
View File

@@ -26,46 +26,57 @@ func TestValidateErrors(t *testing.T) {
ErrorSubstring: "missing commands",
},
{
Name: "package_missing_default_branch",
Item: &Package{
Name: "project_missing_default_branch",
Item: &Project{
Repository: GithubRepository{Owner: "octocat", Name: "hello-world"},
},
ErrorSubstring: "missing default branch",
},
{
Name: "package_missing_version",
Item: &Package{
Name: "project_missing_version",
Item: &Project{
Repository: GithubRepository{Owner: "octocat", Name: "hello-world"},
DefaultBranch: "main",
},
ErrorSubstring: "missing version",
},
{
Name: "package_missing_module",
Item: &Package{
Name: "project_missing_packages",
Item: &Project{
Repository: GithubRepository{Owner: "octocat", Name: "hello-world"},
DefaultBranch: "main",
Version: "v1.0.1",
Version: "v0.1.1",
},
ErrorSubstring: "missing packages",
},
{
Name: "project_package_error",
Item: &Project{
Repository: GithubRepository{Owner: "octocat", Name: "hello-world"},
DefaultBranch: "main",
Version: "v0.1.1",
Packages: []*Package{
{},
},
},
ErrorSubstring: "package root: missing module",
},
{
Name: "package_missing_module",
Item: &Package{},
ErrorSubstring: "missing module",
},
{
Name: "package_no_generate_commands",
Item: &Package{
Repository: GithubRepository{Owner: "octocat", Name: "hello-world"},
DefaultBranch: "main",
Version: "v1.0.1",
Module: "avo/go.mod",
Module: "avo/go.mod",
},
ErrorSubstring: "no generate commands",
},
{
Name: "package_invalid_generate_commands",
Item: &Package{
Repository: GithubRepository{Owner: "octocat", Name: "hello-world"},
DefaultBranch: "main",
Version: "v1.0.1",
Module: "avo/go.mod",
Module: "avo/go.mod",
Generate: []*Step{
{},
},
@@ -73,8 +84,8 @@ func TestValidateErrors(t *testing.T) {
ErrorSubstring: "generate step: missing name",
},
{
Name: "packages_invalid_package",
Item: Packages{
Name: "projects_invalid_package",
Item: Projects{
{
Repository: GithubRepository{Owner: "octocat", Name: "hello-world"},
},
@@ -96,17 +107,17 @@ func TestValidateErrors(t *testing.T) {
}
}
func TestLoadPackagesBad(t *testing.T) {
func TestLoadProjectsBad(t *testing.T) {
r := strings.NewReader(`[{"unknown_field": "value"}]`)
_, err := LoadPackages(r)
_, err := LoadProjects(r)
if err == nil {
t.Fatal("expected non-nil error")
}
}
func TestLoadPackagesFileNotExist(t *testing.T) {
pkgs, err := LoadPackagesFile("does_not_exist")
if pkgs != nil {
func TestLoadProjectsFileNotExist(t *testing.T) {
prjs, err := LoadProjectsFile("does_not_exist")
if prjs != nil {
t.Fatal("expected nil return")
}
if err == nil {
@@ -114,24 +125,24 @@ func TestLoadPackagesFileNotExist(t *testing.T) {
}
}
func TestPackagesFileValid(t *testing.T) {
pkgs, err := LoadPackagesFile("packages.json")
func TestProjectsFileValid(t *testing.T) {
prjs, err := LoadProjectsFile("projects.json")
if err != nil {
t.Fatal(err)
}
for _, pkg := range pkgs {
t.Logf("read: %s", pkg.ID())
for _, prj := range prjs {
t.Logf("read: %s", prj.ID())
}
if len(pkgs) == 0 {
if len(prjs) == 0 {
t.Fatal("no packages loaded")
}
if err := pkgs.Validate(); err != nil {
if err := prjs.Validate(); err != nil {
t.Fatal(err)
}
}
func TestPackagesFileStepsValid(t *testing.T) {
pkgs, err := LoadPackagesFile("packages.json")
func TestProjectsFileStepsValid(t *testing.T) {
prjs, err := LoadProjectsFile("projects.json")
if err != nil {
t.Fatal(err)
}
@@ -139,34 +150,36 @@ func TestPackagesFileStepsValid(t *testing.T) {
AvoDirectory: "avo",
RepositoryDirectory: "repo",
}
for _, pkg := range pkgs {
for _, s := range pkg.Steps(c) {
if err := s.Validate(); err != nil {
t.Errorf("package %s: %s", pkg.ID(), err)
for _, prj := range prjs {
for _, pkg := range prj.Packages {
for _, s := range pkg.Steps(c) {
if err := s.Validate(); err != nil {
t.Errorf("project %s: package %s: %s", prj.ID(), pkg.Name(), err)
}
}
}
}
}
func TestPackagesFileRoundtrip(t *testing.T) {
pkgs, err := LoadPackagesFile("packages.json")
func TestProjectsFileRoundtrip(t *testing.T) {
prjs, err := LoadProjectsFile("projects.json")
if err != nil {
t.Fatal(err)
}
// Write and read back.
buf := bytes.NewBuffer(nil)
if err := StorePackages(buf, pkgs); err != nil {
if err := StoreProjects(buf, prjs); err != nil {
t.Fatal(err)
}
roundtrip, err := LoadPackages(buf)
roundtrip, err := LoadProjects(buf)
if err != nil {
t.Fatal(err)
}
// Should be identical.
if !reflect.DeepEqual(pkgs, roundtrip) {
if !reflect.DeepEqual(prjs, roundtrip) {
t.Fatal("roundtrip mismatch")
}
}

32
tests/thirdparty/make_workflow.go vendored
View File

@@ -16,7 +16,7 @@ import (
)
var (
pkgsfilename = flag.String("pkgs", "", "packages configuration")
prjsfilename = flag.String("prjs", "", "projects configuration")
output = flag.String("output", "", "path to output file (default stdout)")
)
@@ -29,13 +29,13 @@ func main() {
func mainerr() error {
flag.Parse()
// Read packages.
pkgs, err := thirdparty.LoadPackagesFile(*pkgsfilename)
// Read projects.
prjs, err := thirdparty.LoadProjectsFile(*prjsfilename)
if err != nil {
return err
}
if err := pkgs.Validate(); err != nil {
if err := prjs.Validate(); err != nil {
return err
}
@@ -51,7 +51,7 @@ func mainerr() error {
}
// Generate workflow file.
b, err := GenerateWorkflow(pkgs)
b, err := GenerateWorkflow(prjs)
if err != nil {
return err
}
@@ -64,7 +64,7 @@ func mainerr() error {
return nil
}
func GenerateWorkflow(pkgs thirdparty.Packages) ([]byte, error) {
func GenerateWorkflow(prjs thirdparty.Projects) ([]byte, error) {
g := &prnt.Generator{}
g.SetIndentString(" ")
@@ -84,16 +84,16 @@ func GenerateWorkflow(pkgs thirdparty.Packages) ([]byte, error) {
g.Linef(" - master")
g.Linef(" pull_request:")
// One job per package.
// One job per test case.
g.Linef("jobs:")
g.Indent()
for _, pkg := range pkgs {
g.Linef("%s:", pkg.ID())
for _, t := range prjs.Tests() {
g.Linef("%s:", t.ID())
g.Indent()
g.Linef("runs-on: ubuntu-latest")
if pkg.Skip() {
g.Linef("if: false # skip: %s", pkg.Reason())
if t.Project.Skip() {
g.Linef("if: false # skip: %s", t.Project.Reason())
}
g.Linef("steps:")
g.Indent()
@@ -114,12 +114,12 @@ func GenerateWorkflow(pkgs thirdparty.Packages) ([]byte, error) {
g.Linef(" persist-credentials: false")
// Checkout the third-party package.
pkgdir := pkg.Repository.Name
g.Linef("- name: Checkout %s", pkg.Repository)
pkgdir := t.Project.Repository.Name
g.Linef("- name: Checkout %s", t.Project.Repository)
g.Linef(" uses: actions/checkout@5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f # v2.3.4")
g.Linef(" with:")
g.Linef(" repository: %s", pkg.Repository)
g.Linef(" ref: %s", pkg.Version)
g.Linef(" repository: %s", t.Project.Repository)
g.Linef(" ref: %s", t.Project.Version)
g.Linef(" path: %s", pkgdir)
g.Linef(" persist-credentials: false")
@@ -129,7 +129,7 @@ func GenerateWorkflow(pkgs thirdparty.Packages) ([]byte, error) {
RepositoryDirectory: path.Join("${{ github.workspace }}", pkgdir),
}
for _, step := range pkg.Steps(c) {
for _, step := range t.Package.Steps(c) {
g.Linef("- name: %s", step.Name)
g.Linef(" working-directory: %s", path.Join(pkgdir, step.WorkingDirectory))
if len(step.Commands) == 1 {

50
tests/thirdparty/metadata_test.go vendored
View File

@@ -9,84 +9,84 @@ import (
"github.com/mmcloughlin/avo/internal/test"
)
var update = flag.Bool("update", false, "update package metadata")
var update = flag.Bool("update", false, "update project metadata")
func TestPackagesFileMetadata(t *testing.T) {
func TestProjectsFileMetadata(t *testing.T) {
test.RequiresNetwork(t)
ctx := context.Background()
pkgs, err := LoadPackagesFile("packages.json")
prjs, err := LoadProjectsFile("projects.json")
if err != nil {
t.Fatal(err)
}
g := github.NewClient(github.WithTokenFromEnvironment())
for _, pkg := range pkgs {
for _, prj := range prjs {
// Fetch metadata.
r, err := g.Repository(ctx, pkg.Repository.Owner, pkg.Repository.Name)
r, err := g.Repository(ctx, prj.Repository.Owner, prj.Repository.Name)
if err != nil {
t.Fatal(err)
}
// Update, if requested.
if *update {
pkg.DefaultBranch = r.DefaultBranch
pkg.Metadata.Description = r.Description
pkg.Metadata.Homepage = r.Homepage
pkg.Metadata.Stars = r.StargazersCount
prj.DefaultBranch = r.DefaultBranch
prj.Metadata.Description = r.Description
prj.Metadata.Homepage = r.Homepage
prj.Metadata.Stars = r.StargazersCount
t.Logf("%s: metadata updated", pkg.ID())
t.Logf("%s: metadata updated", prj.ID())
}
// Check up to date. Potentially fast-changing properties not included.
uptodate := true
uptodate = pkg.DefaultBranch == r.DefaultBranch && uptodate
uptodate = pkg.Metadata.Description == r.Description && uptodate
uptodate = pkg.Metadata.Homepage == r.Homepage && uptodate
uptodate = prj.DefaultBranch == r.DefaultBranch && uptodate
uptodate = prj.Metadata.Description == r.Description && uptodate
uptodate = prj.Metadata.Homepage == r.Homepage && uptodate
if !uptodate {
t.Errorf("%s: metadata out of date (use -update flag to fix)", pkg.ID())
t.Errorf("%s: metadata out of date (use -update flag to fix)", prj.ID())
}
}
if err := StorePackagesFile("packages.json", pkgs); err != nil {
if err := StoreProjectsFile("projects.json", prjs); err != nil {
t.Fatal(err)
}
}
func TestPackagesFileKnownIssues(t *testing.T) {
func TestProjectsFileKnownIssues(t *testing.T) {
test.RequiresNetwork(t)
ctx := context.Background()
pkgs, err := LoadPackagesFile("packages.json")
prjs, err := LoadProjectsFile("projects.json")
if err != nil {
t.Fatal(err)
}
g := github.NewClient(github.WithTokenFromEnvironment())
for _, pkg := range pkgs {
for _, prj := range prjs {
// Skipped packages must refer to an open issue.
if !pkg.Skip() {
if !prj.Skip() {
continue
}
if pkg.KnownIssue == 0 {
t.Errorf("%s: skipped package must refer to known issue", pkg.ID())
if prj.KnownIssue == 0 {
t.Errorf("%s: skipped package must refer to known issue", prj.ID())
}
issue, err := g.Issue(ctx, "mmcloughlin", "avo", pkg.KnownIssue)
issue, err := g.Issue(ctx, "mmcloughlin", "avo", prj.KnownIssue)
if err != nil {
t.Fatal(err)
}
if issue.State != "open" {
t.Errorf("%s: known issue in %s state", pkg.ID(), issue.State)
t.Errorf("%s: known issue in %s state", prj.ID(), issue.State)
}
if pkg.Reason() != issue.HTMLURL {
t.Errorf("%s: expected skip reason to be the issue url %s", pkg.ID(), issue.HTMLURL)
if prj.Reason() != issue.HTMLURL {
t.Errorf("%s: expected skip reason to be the issue url %s", prj.ID(), issue.HTMLURL)
}
}
}

685
tests/thirdparty/packages.json vendored
View File

@@ -1,685 +0,0 @@
[
{
"repository": {
"owner": "zeebo",
"name": "xxh3"
},
"metadata": {
"description": "XXH3 algorithm in Go",
"stars": 238
},
"default_branch": "master",
"version": "v1.0.0-rc1",
"module": "avo/go.mod",
"generate": [
{
"dir": "avo",
"commands": [
"go run . -avx -out ../accum_vector_avx_amd64.s",
"go run . -sse -out ../accum_vector_sse_amd64.s"
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-sip13"
},
"metadata": {
"description": "siphash 1-3",
"stars": 32
},
"default_branch": "master",
"version": "62edffca92457b3a66125c686137cc5f0fe81672",
"module": "_avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-sip13"
]
},
{
"name": "Setup Generator Module",
"dir": "_avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-sip13/_avo"
]
}
],
"generate": [
{
"dir": "_avo",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/dgryski/go-sip13@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/dgryski/go-sip13=../",
"go run . -out ../sip13_amd64.s"
]
}
]
},
{
"repository": {
"owner": "phoreproject",
"name": "bls"
},
"metadata": {
"description": "Go implementation of the BLS12-381 pairing",
"stars": 81
},
"default_branch": "master",
"version": "a88a5ae26844d7293359422888d7c7f69f43c845",
"module": "asm/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod tidy"
]
},
{
"name": "Setup Generator Module",
"dir": "asm",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/phoreproject/bls/asm"
]
}
],
"generate": [
{
"dir": "asm",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/phoreproject/bls@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/phoreproject/bls=../",
"go mod download",
"go run . -out ../primitivefuncs_amd64.s"
]
}
]
},
{
"repository": {
"owner": "minio",
"name": "md5-simd"
},
"metadata": {
"description": "Accelerate aggregated MD5 hashing performance up to 8x for AVX512 and 4x for AVX2. Useful for server applications that need to compute many MD5 sums in parallel.",
"stars": 108
},
"default_branch": "master",
"version": "30ad8af83f6868c2a30c615f3edf1a9366bf3f89",
"module": "_gen/go.mod",
"generate": [
{
"dir": "_gen",
"commands": [
"go run gen.go -out ../md5block_amd64.s -stubs ../md5block_amd64.go -pkg md5simd"
]
}
]
},
{
"repository": {
"owner": "zeebo",
"name": "blake3"
},
"metadata": {
"description": "Pure Go implementation of BLAKE3 with AVX2 and SSE4.1 acceleration",
"stars": 288
},
"default_branch": "master",
"version": "25dba572f0e78ec108f0dd79c9c15288f542d7d9",
"module": "avo/go.mod",
"generate": [
{
"dir": "avo",
"commands": [
"go run ./avx2 -out ../avx2/impl.s",
"go run ./sse41 -out ../sse41/impl.s"
]
}
]
},
{
"repository": {
"owner": "klauspost",
"name": "reedsolomon"
},
"metadata": {
"description": "Reed-Solomon Erasure Coding in Go",
"stars": 1437
},
"default_branch": "master",
"version": "922778284547557265cff0f903ab5f4c27e40ae9",
"module": "_gen/go.mod",
"generate": [
{
"dir": "_gen",
"commands": [
"go run gen.go -out ../galois_gen_amd64.s -stubs ../galois_gen_amd64.go -pkg reedsolomon"
]
}
]
},
{
"repository": {
"owner": "orisano",
"name": "wyhash"
},
"metadata": {
"description": "A pure-Go wyhash implementation.",
"stars": 24
},
"default_branch": "master",
"version": "32a3f7f6ba4797e2d87dab2969cc9dd63d39cce9",
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"go mod init github.com/orisano/wyhash/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go run gen.go -out ../blocks_amd64.s -stubs ../blocks_amd64.go -pkg wyhash"
]
}
]
},
{
"repository": {
"owner": "klauspost",
"name": "compress"
},
"metadata": {
"description": "Optimized Go Compression Packages",
"stars": 2703
},
"default_branch": "master",
"version": "2adf487b3e02f95ce7efd6e4953fda0ae7ecd080",
"pkg": "s2",
"module": "_generate/go.mod",
"generate": [
{
"dir": "_generate",
"commands": [
"go run gen.go -out ../encodeblock_amd64.s -stubs ../encodeblock_amd64.go -pkg s2"
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-bloomindex"
},
"metadata": {
"description": "Bloom-filter based search index",
"stars": 111
},
"default_branch": "master",
"version": "0902316dce158c154b958ee5cfc706c62af29a42",
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-bloomindex",
"go get github.com/dgryski/go-bits@v0.0.0-20180113010104-bd8a69a71dc2"
]
},
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv asm.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-bloomindex/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go run asm.go -out ../query_amd64.s"
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-marvin32"
},
"metadata": {
"description": "Assembly-optimized Marvin32 hash function",
"stars": 12
},
"default_branch": "master",
"version": "7d18f4c6ea7c24b29d1c7a670f8ae40b0812f2e3",
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-marvin32"
]
},
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv asm.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-marvin32/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/dgryski/go-marvin32@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/dgryski/go-marvin32=../",
"go run asm.go -out ../marvin_amd64.s"
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-speck"
},
"metadata": {
"description": "SPECK cipher",
"stars": 10
},
"default_branch": "master",
"version": "5b36d4c08d8840c352a153bf37281434ad550ec0",
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-speck",
"go get github.com/dgryski/go-skipjack@v0.0.0-20150401095227-f3a984a142ab"
]
},
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv asm.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-speck/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/dgryski/go-speck@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/dgryski/go-speck=../",
"go run asm.go -out ../speck_amd64.s"
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-chaskey"
},
"metadata": {
"description": "go-chaskey: an implementation of chaskey, an efficient MAC for microcontrollers",
"stars": 8
},
"default_branch": "master",
"version": "ba454392bc5ab6daae103e15147185f8f4a27dcc",
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-chaskey",
"go get github.com/dchest/siphash@v1.2.2"
]
},
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv asm.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-chaskey/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/dgryski/go-chaskey@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/dgryski/go-chaskey=../",
"go run asm.go -out ../core_amd64.s"
]
}
]
},
{
"repository": {
"owner": "lukechampine",
"name": "us"
},
"metadata": {
"description": "An alternative interface to Sia",
"stars": 52
},
"default_branch": "master",
"version": "dff56a80f83653cb14eeeb57ba6ba3c3e942c412",
"pkg": "merkle/blake2b",
"module": "avo/go.mod",
"setup": [
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv gen.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' gen.go",
"go mod init github.com/lukechampine/us/merkle/blake2b/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go run gen.go -out ../blake2b_amd64.s"
]
}
]
},
{
"repository": {
"owner": "segmentio",
"name": "asm"
},
"metadata": {
"description": "Go library providing algorithms optimized to leverage the characteristics of modern CPUs",
"stars": 685
},
"default_branch": "main",
"version": "2bf3fdd47b1cdbe8fe020f032caf14891e10a85f",
"module": "build/go.mod",
"generate": [
{
"commands": [
"make --always-make build"
]
}
]
},
{
"repository": {
"owner": "ericlagergren",
"name": "lwcrypto"
},
"metadata": {
"description": "NIST Lightweight Cryptography finalists",
"stars": 4
},
"default_branch": "main",
"version": "0c42b05eddc34c58bf8e0cd4250c5cd2c256ea57",
"pkg": "ascon",
"module": "asm/go.mod",
"generate": [
{
"dir": "asm",
"commands": [
"go mod edit -require=github.com/ericlagergren/lwcrypto@v0.0.0-00010101000000-000000000000",
"go generate -v -x"
]
}
]
},
{
"repository": {
"owner": "ericlagergren",
"name": "lwcrypto"
},
"metadata": {
"description": "NIST Lightweight Cryptography finalists",
"stars": 4
},
"default_branch": "main",
"version": "0c42b05eddc34c58bf8e0cd4250c5cd2c256ea57",
"pkg": "grain",
"module": "asm/go.mod",
"generate": [
{
"dir": "asm",
"commands": [
"go mod edit -require=github.com/ericlagergren/lwcrypto@v0.0.0-00010101000000-000000000000",
"go generate -v -x"
]
}
]
},
{
"repository": {
"owner": "oasisprotocol",
"name": "curve25519-voi"
},
"metadata": {
"description": "High-performance Curve25519/ristretto255 for Go",
"stars": 45
},
"default_branch": "master",
"version": "d5a936accd94ef9da4c0fe9db0a6342dcdcfeadf",
"module": "internal/asm/amd64/go.mod",
"generate": [
{
"dir": "internal/asm/amd64",
"commands": [
"bash -ex gen.sh"
]
}
]
},
{
"repository": {
"owner": "oasisprotocol",
"name": "deoxysii"
},
"metadata": {
"description": "Go Deoxys-II-256-128",
"stars": 17
},
"default_branch": "master",
"version": "2091330c22b71dd4e04e15306498e68f3a1b3afc",
"module": "internal/hardware/asm/go.mod",
"generate": [
{
"dir": "internal/hardware/asm",
"commands": [
"bash -ex gen.sh"
]
}
]
},
{
"repository": {
"owner": "golang",
"name": "crypto"
},
"metadata": {
"description": "[mirror] Go supplementary cryptography libraries",
"homepage": "https://golang.org/x/crypto",
"stars": 2402
},
"default_branch": "master",
"version": "089bfa5675191fd96a44247682f76ebca03d7916",
"pkg": "curve25519",
"module": "internal/field/_asm/go.mod",
"generate": [
{
"dir": "internal/field/_asm",
"commands": [
"go generate -v -x"
]
}
]
},
{
"repository": {
"owner": "FiloSottile",
"name": "edwards25519"
},
"metadata": {
"description": "filippo.io/edwards25519 — A safer, faster, and more powerful low-level edwards25519 Go implementation.",
"homepage": "https://filippo.io/edwards25519",
"stars": 81
},
"default_branch": "main",
"version": "v1.0.0-rc.1",
"module": "field/_asm/go.mod",
"generate": [
{
"dir": "field/_asm",
"commands": [
"go generate -v -x"
]
}
]
},
{
"repository": {
"owner": "lukechampine",
"name": "blake3"
},
"metadata": {
"description": "A pure-Go implementation of the BLAKE3 cryptographic hash function",
"stars": 278
},
"default_branch": "master",
"version": "v1.1.7",
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/build ignore/d' gen.go",
"go mod init github.com/lukechampine/blake3/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go run gen.go -out ../blake3_amd64.s"
]
}
]
},
{
"repository": {
"owner": "mmcloughlin",
"name": "md4"
},
"metadata": {
"description": "Assembly-optimized MD4 hash algorithm in Go",
"stars": 15
},
"default_branch": "main",
"version": "v0.1.2",
"module": "asm/go.mod",
"generate": [
{
"commands": [
"go generate -v -x"
]
}
]
},
{
"repository": {
"owner": "golang",
"name": "go"
},
"metadata": {
"description": "The Go programming language",
"homepage": "https://go.dev",
"stars": 98269
},
"default_branch": "master",
"version": "go1.17.3",
"module": "src/crypto/ed25519/internal/edwards25519/field/_asm/go.mod",
"setup": [
{
"name": "Compile Go Toolchain",
"dir": "src",
"commands": [
"./make.bash"
]
}
],
"generate": [
{
"dir": "src/crypto/ed25519/internal/edwards25519/field/_asm",
"commands": [
"go generate -v -x"
]
}
],
"test": [
{
"commands": [
"env --unset=GOROOT ./bin/go test crypto/ed25519/..."
]
}
]
}
]

40
tests/thirdparty/packages_test.go vendored
View File

@@ -10,33 +10,33 @@ import (
"github.com/mmcloughlin/avo/internal/test"
)
//go:generate go run make_workflow.go -pkgs packages.json -output ../../.github/workflows/packages.yml
//go:generate go run make_workflow.go -prjs projects.json -output ../../.github/workflows/packages.yml
// Custom flags.
var (
pkgsfilename = flag.String("pkgs", "", "packages configuration")
prjsfilename = flag.String("prjs", "", "projects configuration")
preserve = flag.Bool("preserve", false, "preserve working directories")
latest = flag.Bool("latest", false, "use latest versions of each package")
latest = flag.Bool("latest", false, "use latest versions of each project")
)
// TestPackages runs integration tests on all packages specified by packages
// TestPackages runs integration tests on all packages specified by projects
// file given on the command line.
func TestPackages(t *testing.T) {
// Load packages.
if *pkgsfilename == "" {
t.Skip("no packages specified")
// Load projects.
if *prjsfilename == "" {
t.Skip("no projects specified")
}
pkgs, err := LoadPackagesFile(*pkgsfilename)
prjs, err := LoadProjectsFile(*prjsfilename)
if err != nil {
t.Fatal(err)
}
for _, pkg := range pkgs {
pkg := pkg // scopelint
t.Run(pkg.ID(), func(t *testing.T) {
if pkg.Skip() {
t.Skipf("skip: %s", pkg.Reason())
for _, tst := range prjs.Tests() {
tst := tst // scopelint
t.Run(tst.ID(), func(t *testing.T) {
if tst.Project.Skip() {
t.Skipf("skip: %s", tst.Project.Reason())
}
dir, clean := test.TempDir(t)
if !*preserve {
@@ -46,7 +46,7 @@ func TestPackages(t *testing.T) {
}
pt := PackageTest{
T: t,
Package: pkg,
Test: tst,
WorkDir: dir,
Latest: *latest,
}
@@ -58,10 +58,10 @@ func TestPackages(t *testing.T) {
// PackageTest executes an integration test based on a given third-party package.
type PackageTest struct {
*testing.T
*Package
*Test
WorkDir string // working directory for the test
Latest bool // use latest version of the package
Latest bool // use latest version of the project
repopath string // path the repo is cloned to
}
@@ -75,15 +75,15 @@ func (t *PackageTest) Run() {
// checkout the code at the specified version.
func (t *PackageTest) checkout() {
// Determine the version we want to checkout.
version := t.Version
version := t.Project.Version
if t.Latest {
version = t.DefaultBranch
version = t.Project.DefaultBranch
}
// Clone. Use a shallow clone to speed up large repositories.
t.repopath = filepath.Join(t.WorkDir, t.Name())
test.Exec(t.T, "git", "init", t.repopath)
test.Exec(t.T, "git", "-C", t.repopath, "remote", "add", "origin", t.Repository.CloneURL())
test.Exec(t.T, "git", "-C", t.repopath, "remote", "add", "origin", t.Project.Repository.CloneURL())
test.Exec(t.T, "git", "-C", t.repopath, "fetch", "--depth=1", "origin", version)
test.Exec(t.T, "git", "-C", t.repopath, "checkout", "FETCH_HEAD")
}
@@ -102,7 +102,7 @@ func (t *PackageTest) steps() {
RepositoryDirectory: t.repopath,
}
for _, s := range t.Steps(c) {
for _, s := range t.Package.Steps(c) {
for _, command := range s.Commands {
cmd := exec.Command("sh", "-c", command)
cmd.Dir = filepath.Join(t.repopath, s.WorkingDirectory)

763
tests/thirdparty/projects.json vendored Normal file
View File

@@ -0,0 +1,763 @@
[
{
"repository": {
"owner": "zeebo",
"name": "xxh3"
},
"metadata": {
"description": "XXH3 algorithm in Go",
"stars": 238
},
"default_branch": "master",
"version": "v1.0.0-rc1",
"packages": [
{
"module": "avo/go.mod",
"generate": [
{
"dir": "avo",
"commands": [
"go run . -avx -out ../accum_vector_avx_amd64.s",
"go run . -sse -out ../accum_vector_sse_amd64.s"
]
}
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-sip13"
},
"metadata": {
"description": "siphash 1-3",
"stars": 32
},
"default_branch": "master",
"version": "62edffca92457b3a66125c686137cc5f0fe81672",
"packages": [
{
"module": "_avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-sip13"
]
},
{
"name": "Setup Generator Module",
"dir": "_avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-sip13/_avo"
]
}
],
"generate": [
{
"dir": "_avo",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/dgryski/go-sip13@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/dgryski/go-sip13=../",
"go run . -out ../sip13_amd64.s"
]
}
]
}
]
},
{
"repository": {
"owner": "phoreproject",
"name": "bls"
},
"metadata": {
"description": "Go implementation of the BLS12-381 pairing",
"stars": 81
},
"default_branch": "master",
"version": "a88a5ae26844d7293359422888d7c7f69f43c845",
"packages": [
{
"module": "asm/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod tidy"
]
},
{
"name": "Setup Generator Module",
"dir": "asm",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/phoreproject/bls/asm"
]
}
],
"generate": [
{
"dir": "asm",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/phoreproject/bls@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/phoreproject/bls=../",
"go mod download",
"go run . -out ../primitivefuncs_amd64.s"
]
}
]
}
]
},
{
"repository": {
"owner": "minio",
"name": "md5-simd"
},
"metadata": {
"description": "Accelerate aggregated MD5 hashing performance up to 8x for AVX512 and 4x for AVX2. Useful for server applications that need to compute many MD5 sums in parallel.",
"stars": 109
},
"default_branch": "master",
"version": "30ad8af83f6868c2a30c615f3edf1a9366bf3f89",
"packages": [
{
"module": "_gen/go.mod",
"generate": [
{
"dir": "_gen",
"commands": [
"go run gen.go -out ../md5block_amd64.s -stubs ../md5block_amd64.go -pkg md5simd"
]
}
]
}
]
},
{
"repository": {
"owner": "zeebo",
"name": "blake3"
},
"metadata": {
"description": "Pure Go implementation of BLAKE3 with AVX2 and SSE4.1 acceleration",
"stars": 288
},
"default_branch": "master",
"version": "25dba572f0e78ec108f0dd79c9c15288f542d7d9",
"packages": [
{
"module": "avo/go.mod",
"generate": [
{
"dir": "avo",
"commands": [
"go run ./avx2 -out ../avx2/impl.s",
"go run ./sse41 -out ../sse41/impl.s"
]
}
]
}
]
},
{
"repository": {
"owner": "klauspost",
"name": "reedsolomon"
},
"metadata": {
"description": "Reed-Solomon Erasure Coding in Go",
"stars": 1438
},
"default_branch": "master",
"version": "922778284547557265cff0f903ab5f4c27e40ae9",
"packages": [
{
"module": "_gen/go.mod",
"generate": [
{
"dir": "_gen",
"commands": [
"go run gen.go -out ../galois_gen_amd64.s -stubs ../galois_gen_amd64.go -pkg reedsolomon"
]
}
]
}
]
},
{
"repository": {
"owner": "orisano",
"name": "wyhash"
},
"metadata": {
"description": "A pure-Go wyhash implementation.",
"stars": 24
},
"default_branch": "master",
"version": "32a3f7f6ba4797e2d87dab2969cc9dd63d39cce9",
"packages": [
{
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"go mod init github.com/orisano/wyhash/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go run gen.go -out ../blocks_amd64.s -stubs ../blocks_amd64.go -pkg wyhash"
]
}
]
}
]
},
{
"repository": {
"owner": "klauspost",
"name": "compress"
},
"metadata": {
"description": "Optimized Go Compression Packages",
"stars": 2706
},
"default_branch": "master",
"version": "2adf487b3e02f95ce7efd6e4953fda0ae7ecd080",
"packages": [
{
"pkg": "s2",
"module": "_generate/go.mod",
"generate": [
{
"dir": "_generate",
"commands": [
"go run gen.go -out ../encodeblock_amd64.s -stubs ../encodeblock_amd64.go -pkg s2"
]
}
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-bloomindex"
},
"metadata": {
"description": "Bloom-filter based search index",
"stars": 111
},
"default_branch": "master",
"version": "0902316dce158c154b958ee5cfc706c62af29a42",
"packages": [
{
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-bloomindex",
"go get github.com/dgryski/go-bits@v0.0.0-20180113010104-bd8a69a71dc2"
]
},
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv asm.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-bloomindex/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go run asm.go -out ../query_amd64.s"
]
}
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-marvin32"
},
"metadata": {
"description": "Assembly-optimized Marvin32 hash function",
"stars": 12
},
"default_branch": "master",
"version": "7d18f4c6ea7c24b29d1c7a670f8ae40b0812f2e3",
"packages": [
{
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-marvin32"
]
},
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv asm.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-marvin32/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/dgryski/go-marvin32@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/dgryski/go-marvin32=../",
"go run asm.go -out ../marvin_amd64.s"
]
}
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-speck"
},
"metadata": {
"description": "SPECK cipher",
"stars": 10
},
"default_branch": "master",
"version": "5b36d4c08d8840c352a153bf37281434ad550ec0",
"packages": [
{
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-speck",
"go get github.com/dgryski/go-skipjack@v0.0.0-20150401095227-f3a984a142ab"
]
},
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv asm.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-speck/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/dgryski/go-speck@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/dgryski/go-speck=../",
"go run asm.go -out ../speck_amd64.s"
]
}
]
}
]
},
{
"repository": {
"owner": "dgryski",
"name": "go-chaskey"
},
"metadata": {
"description": "go-chaskey: an implementation of chaskey, an efficient MAC for microcontrollers",
"stars": 8
},
"default_branch": "master",
"version": "ba454392bc5ab6daae103e15147185f8f4a27dcc",
"packages": [
{
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Root Module",
"commands": [
"go mod init github.com/dgryski/go-chaskey",
"go get github.com/dchest/siphash@v1.2.2"
]
},
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv asm.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' asm.go",
"go mod init github.com/dgryski/go-chaskey/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go mod edit -modfile=go.mod -require=github.com/dgryski/go-chaskey@v0.0.0-00010101000000-000000000000",
"go mod edit -modfile=go.mod -replace=github.com/dgryski/go-chaskey=../",
"go run asm.go -out ../core_amd64.s"
]
}
]
}
]
},
{
"repository": {
"owner": "lukechampine",
"name": "us"
},
"metadata": {
"description": "An alternative interface to Sia",
"stars": 52
},
"default_branch": "master",
"version": "dff56a80f83653cb14eeeb57ba6ba3c3e942c412",
"packages": [
{
"pkg": "merkle/blake2b",
"module": "avo/go.mod",
"setup": [
{
"name": "Create Generator Module",
"commands": [
"mkdir avo",
"git mv gen.go avo"
]
},
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/+build ignore/d' gen.go",
"go mod init github.com/lukechampine/us/merkle/blake2b/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go run gen.go -out ../blake2b_amd64.s"
]
}
]
}
]
},
{
"repository": {
"owner": "segmentio",
"name": "asm"
},
"metadata": {
"description": "Go library providing algorithms optimized to leverage the characteristics of modern CPUs",
"stars": 686
},
"default_branch": "main",
"version": "2bf3fdd47b1cdbe8fe020f032caf14891e10a85f",
"packages": [
{
"module": "build/go.mod",
"generate": [
{
"commands": [
"make --always-make build"
]
}
]
}
]
},
{
"repository": {
"owner": "ericlagergren",
"name": "lwcrypto"
},
"metadata": {
"description": "NIST Lightweight Cryptography finalists",
"stars": 4
},
"default_branch": "main",
"version": "0c42b05eddc34c58bf8e0cd4250c5cd2c256ea57",
"packages": [
{
"pkg": "ascon",
"module": "asm/go.mod",
"generate": [
{
"dir": "asm",
"commands": [
"go mod edit -require=github.com/ericlagergren/lwcrypto@v0.0.0-00010101000000-000000000000",
"go generate -v -x"
]
}
]
},
{
"pkg": "grain",
"module": "asm/go.mod",
"generate": [
{
"dir": "asm",
"commands": [
"go mod edit -require=github.com/ericlagergren/lwcrypto@v0.0.0-00010101000000-000000000000",
"go generate -v -x"
]
}
]
}
]
},
{
"repository": {
"owner": "oasisprotocol",
"name": "curve25519-voi"
},
"metadata": {
"description": "High-performance Curve25519/ristretto255 for Go",
"stars": 45
},
"default_branch": "master",
"version": "d5a936accd94ef9da4c0fe9db0a6342dcdcfeadf",
"packages": [
{
"module": "internal/asm/amd64/go.mod",
"generate": [
{
"dir": "internal/asm/amd64",
"commands": [
"bash -ex gen.sh"
]
}
]
}
]
},
{
"repository": {
"owner": "oasisprotocol",
"name": "deoxysii"
},
"metadata": {
"description": "Go Deoxys-II-256-128",
"stars": 17
},
"default_branch": "master",
"version": "2091330c22b71dd4e04e15306498e68f3a1b3afc",
"packages": [
{
"module": "internal/hardware/asm/go.mod",
"generate": [
{
"dir": "internal/hardware/asm",
"commands": [
"bash -ex gen.sh"
]
}
]
}
]
},
{
"repository": {
"owner": "golang",
"name": "crypto"
},
"metadata": {
"description": "[mirror] Go supplementary cryptography libraries",
"homepage": "https://golang.org/x/crypto",
"stars": 2402
},
"default_branch": "master",
"version": "089bfa5675191fd96a44247682f76ebca03d7916",
"packages": [
{
"pkg": "curve25519",
"module": "internal/field/_asm/go.mod",
"generate": [
{
"dir": "internal/field/_asm",
"commands": [
"go generate -v -x"
]
}
]
}
]
},
{
"repository": {
"owner": "FiloSottile",
"name": "edwards25519"
},
"metadata": {
"description": "filippo.io/edwards25519 — A safer, faster, and more powerful low-level edwards25519 Go implementation.",
"homepage": "https://filippo.io/edwards25519",
"stars": 81
},
"default_branch": "main",
"version": "v1.0.0-rc.1",
"packages": [
{
"module": "field/_asm/go.mod",
"generate": [
{
"dir": "field/_asm",
"commands": [
"go generate -v -x"
]
}
]
}
]
},
{
"repository": {
"owner": "lukechampine",
"name": "blake3"
},
"metadata": {
"description": "A pure-Go implementation of the BLAKE3 cryptographic hash function",
"stars": 277
},
"default_branch": "master",
"version": "v1.1.7",
"packages": [
{
"module": "avo/go.mod",
"setup": [
{
"name": "Setup Generator Module",
"dir": "avo",
"commands": [
"sed -i.bak '/build ignore/d' gen.go",
"go mod init github.com/lukechampine/blake3/avo"
]
}
],
"generate": [
{
"dir": "avo",
"commands": [
"go run gen.go -out ../blake3_amd64.s"
]
}
]
}
]
},
{
"repository": {
"owner": "mmcloughlin",
"name": "md4"
},
"metadata": {
"description": "Assembly-optimized MD4 hash algorithm in Go",
"stars": 15
},
"default_branch": "main",
"version": "v0.1.2",
"packages": [
{
"module": "asm/go.mod",
"generate": [
{
"commands": [
"go generate -v -x"
]
}
]
}
]
},
{
"repository": {
"owner": "golang",
"name": "go"
},
"metadata": {
"description": "The Go programming language",
"homepage": "https://go.dev",
"stars": 98315
},
"default_branch": "master",
"version": "go1.17.3",
"packages": [
{
"module": "src/crypto/ed25519/internal/edwards25519/field/_asm/go.mod",
"setup": [
{
"name": "Compile Go Toolchain",
"dir": "src",
"commands": [
"./make.bash"
]
}
],
"generate": [
{
"dir": "src/crypto/ed25519/internal/edwards25519/field/_asm",
"commands": [
"go generate -v -x"
]
}
],
"test": [
{
"commands": [
"env --unset=GOROOT ./bin/go test crypto/ed25519/..."
]
}
]
}
]
}
]