Skip to content

GCloud

Since testcontainers-go v0.25.0

Introduction

The Testcontainers module for GCloud.

Adding this module to your project dependencies

Please run the following command to add the GCloud module to your Go dependencies:

go get github.com/testcontainers/testcontainers-go/modules/gcloud

Usage example

Info

By default, the all the emulators use gcr.io/google.com/cloudsdktool/cloud-sdk:367.0.0-emulators as the default Docker image, except for the BigQuery emulator, which uses ghcr.io/goccy/bigquery-emulator:0.4.3, and Spanner, which uses gcr.io/cloud-spanner-emulator/emulator:1.4.0.

BigQuery

ctx := context.Background()

bigQueryContainer, err := gcloud.RunBigQueryContainer(
    ctx,
    testcontainers.WithImage("ghcr.io/goccy/bigquery-emulator:0.4.3"),
    gcloud.WithProjectID("bigquery-project"),
)
if err != nil {
    panic(err)
}

// Clean up the container
defer func() {
    if err := bigQueryContainer.Terminate(ctx); err != nil {
        panic(err)
    }
}()

projectID := bigQueryContainer.Settings.ProjectID

opts := []option.ClientOption{
    option.WithEndpoint(bigQueryContainer.URI),
    option.WithGRPCDialOption(grpc.WithTransportCredentials(insecure.NewCredentials())),
    option.WithoutAuthentication(),
    internaloption.SkipDialSettingsValidation(),
}

client, err := bigquery.NewClient(ctx, projectID, opts...)
if err != nil {
    panic(err)
}
defer client.Close()

It's important to set the option.WithEndpoint() option using the container's URI, as shown in the client example above.

BigTable

ctx := context.Background()

bigTableContainer, err := gcloud.RunBigTableContainer(
    ctx,
    testcontainers.WithImage("gcr.io/google.com/cloudsdktool/cloud-sdk:367.0.0-emulators"),
    gcloud.WithProjectID("bigtable-project"),
)
if err != nil {
    panic(err)
}

// Clean up the container
defer func() {
    if err := bigTableContainer.Terminate(ctx); err != nil {
        panic(err)
    }
}()

projectId := bigTableContainer.Settings.ProjectID

const (
    instanceId = "test-instance"
    tableName  = "test-table"
)

options := []option.ClientOption{
    option.WithEndpoint(bigTableContainer.URI),
    option.WithoutAuthentication(),
    option.WithGRPCDialOption(grpc.WithTransportCredentials(insecure.NewCredentials())),
}
adminClient, err := bigtable.NewAdminClient(ctx, projectId, instanceId, options...)
if err != nil {
    panic(err)
}
defer adminClient.Close()

client, err := bigtable.NewClient(ctx, projectId, instanceId, options...)
if err != nil {
    panic(err)
}
defer client.Close()

It's important to set the option.WithEndpoint() option using the container's URI, as shown in the Admin client example above.

Datastore

ctx := context.Background()

datastoreContainer, err := gcloud.RunDatastoreContainer(
    ctx,
    testcontainers.WithImage("gcr.io/google.com/cloudsdktool/cloud-sdk:367.0.0-emulators"),
    gcloud.WithProjectID("datastore-project"),
)
if err != nil {
    panic(err)
}

// Clean up the container
defer func() {
    if err := datastoreContainer.Terminate(ctx); err != nil {
        panic(err)
    }
}()

projectID := datastoreContainer.Settings.ProjectID

options := []option.ClientOption{
    option.WithEndpoint(datastoreContainer.URI),
    option.WithoutAuthentication(),
    option.WithGRPCDialOption(grpc.WithTransportCredentials(insecure.NewCredentials())),
}

dsClient, err := datastore.NewClient(ctx, projectID, options...)
if err != nil {
    panic(err)
}
defer dsClient.Close()

It's important to set the option.WithEndpoint() option using the container's URI, as shown in the client example above.

Firestore

ctx := context.Background()

firestoreContainer, err := gcloud.RunFirestoreContainer(
    ctx,
    testcontainers.WithImage("gcr.io/google.com/cloudsdktool/cloud-sdk:367.0.0-emulators"),
    gcloud.WithProjectID("firestore-project"),
)
if err != nil {
    panic(err)
}

// Clean up the container
defer func() {
    if err := firestoreContainer.Terminate(ctx); err != nil {
        panic(err)
    }
}()

projectID := firestoreContainer.Settings.ProjectID

conn, err := grpc.Dial(firestoreContainer.URI, grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithPerRPCCredentials(emulatorCreds{}))
if err != nil {
    panic(err)
}

options := []option.ClientOption{option.WithGRPCConn(conn)}
client, err := firestore.NewClient(ctx, projectID, options...)
if err != nil {
    panic(err)
}
defer client.Close()

It's important to set the target string of the grpc.Dial method using the container's URI, as shown in the client example above.

Pubsub

ctx := context.Background()

pubsubContainer, err := gcloud.RunPubsubContainer(
    ctx,
    testcontainers.WithImage("gcr.io/google.com/cloudsdktool/cloud-sdk:367.0.0-emulators"),
    gcloud.WithProjectID("pubsub-project"),
)
if err != nil {
    panic(err)
}

// Clean up the container
defer func() {
    if err := pubsubContainer.Terminate(ctx); err != nil {
        panic(err)
    }
}()

projectID := pubsubContainer.Settings.ProjectID

conn, err := grpc.Dial(pubsubContainer.URI, grpc.WithTransportCredentials(insecure.NewCredentials()))
if err != nil {
    panic(err)
}

options := []option.ClientOption{option.WithGRPCConn(conn)}
client, err := pubsub.NewClient(ctx, projectID, options...)
if err != nil {
    panic(err)
}
defer client.Close()

It's important to set the target string of the grpc.Dial method using the container's URI, as shown in the client example above.

Spanner

ctx := context.Background()

spannerContainer, err := gcloud.RunSpannerContainer(
    ctx,
    testcontainers.WithImage("gcr.io/cloud-spanner-emulator/emulator:1.4.0"),
    gcloud.WithProjectID("spanner-project"),
)
if err != nil {
    panic(err)
}

// Clean up the container
defer func() {
    if err := spannerContainer.Terminate(ctx); err != nil {
        panic(err)
    }
}()

projectId := spannerContainer.Settings.ProjectID

const (
    instanceId   = "test-instance"
    databaseName = "test-db"
)

options := []option.ClientOption{
    option.WithEndpoint(spannerContainer.URI),
    option.WithGRPCDialOption(grpc.WithTransportCredentials(insecure.NewCredentials())),
    option.WithoutAuthentication(),
    internaloption.SkipDialSettingsValidation(),
}

instanceAdmin, err := instance.NewInstanceAdminClient(ctx, options...)
if err != nil {
    panic(err)
}
defer instanceAdmin.Close()

c, err := database.NewDatabaseAdminClient(ctx, options...)
if err != nil {
    panic(err)
}
defer c.Close()

It's important to set the option.WithEndpoint() option using the container's URI, as shown in the Admin client example above.

Module reference

The GCloud module exposes one entrypoint function to create the different GCloud emulators, and each function receives two parameters:

func RunBigQueryContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*BigQueryContainer, error)
func RunBigTableContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*BigTableContainer, error)
func RunDatastoreContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*DatastoreContainer, error)
func RunFirestoreContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*FirestoreContainer, error)
func RunPubsubContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*PubsubContainer, error)
func RunSpannerContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*SpannerContainer, error)
  • context.Context, the Go context.
  • testcontainers.ContainerCustomizer, a variadic argument for passing options.

Container Options

When starting any of the GCloud containers, you can pass options in a variadic way to configure it.

Image

If you need to set a different GCloud Docker image, you can use testcontainers.WithImage with a valid Docker image for GCloud. E.g. testcontainers.WithImage("gcr.io/google.com/cloudsdktool/cloud-sdk:367.0.0-emulators").

Image Substitutions

In more locked down / secured environments, it can be problematic to pull images from Docker Hub and run them without additional precautions.

An image name substitutor converts a Docker image name, as may be specified in code, to an alternative name. This is intended to provide a way to override image names, for example to enforce pulling of images from a private registry.

Testcontainers for Go exposes an interface to perform this operations: ImageSubstitutor, and a No-operation implementation to be used as reference for custom implementations:

package testcontainers

import (
    "context"
    "errors"
    "fmt"
    "sync"

    "github.com/testcontainers/testcontainers-go/internal/testcontainersdocker"
    "github.com/testcontainers/testcontainers-go/internal/testcontainerssession"
)

var (
    reuseContainerMx  sync.Mutex
    ErrReuseEmptyName = errors.New("with reuse option a container name mustn't be empty")
)

// GenericContainerRequest represents parameters to a generic container
type GenericContainerRequest struct {
    ContainerRequest              // embedded request for provider
    Started          bool         // whether to auto-start the container
    ProviderType     ProviderType // which provider to use, Docker if empty
    Logger           Logging      // provide a container specific Logging - use default global logger if empty
    Reuse            bool         // reuse an existing container if it exists or create a new one. a container name mustn't be empty
}

// GenericNetworkRequest represents parameters to a generic network
type GenericNetworkRequest struct {
    NetworkRequest              // embedded request for provider
    ProviderType   ProviderType // which provider to use, Docker if empty
}

// GenericNetwork creates a generic network with parameters
func GenericNetwork(ctx context.Context, req GenericNetworkRequest) (Network, error) {
    provider, err := req.ProviderType.GetProvider()
    if err != nil {
        return nil, err
    }
    network, err := provider.CreateNetwork(ctx, req.NetworkRequest)
    if err != nil {
        return nil, fmt.Errorf("%w: failed to create network", err)
    }

    return network, nil
}

// GenericContainer creates a generic container with parameters
func GenericContainer(ctx context.Context, req GenericContainerRequest) (Container, error) {
    if req.Reuse && req.Name == "" {
        return nil, ErrReuseEmptyName
    }

    logging := req.Logger
    if logging == nil {
        logging = Logger
    }
    provider, err := req.ProviderType.GetProvider(WithLogger(logging))
    if err != nil {
        return nil, err
    }
    defer provider.Close()

    var c Container
    if req.Reuse {
        // we must protect the reusability of the container in the case it's invoked
        // in a parallel execution, via ParallelContainers or t.Parallel()
        reuseContainerMx.Lock()
        defer reuseContainerMx.Unlock()

        c, err = provider.ReuseOrCreateContainer(ctx, req.ContainerRequest)
    } else {
        c, err = provider.CreateContainer(ctx, req.ContainerRequest)
    }
    if err != nil {
        return nil, fmt.Errorf("%w: failed to create container", err)
    }

    if req.Started && !c.IsRunning() {
        if err := c.Start(ctx); err != nil {
            return c, fmt.Errorf("%w: failed to start container", err)
        }
    }
    return c, nil
}

// GenericProvider represents an abstraction for container and network providers
type GenericProvider interface {
    ContainerProvider
    NetworkProvider
    ImageProvider
}

// GenericLabels returns a map of labels that can be used to identify containers created by this library
func GenericLabels() map[string]string {
    return testcontainersdocker.DefaultLabels(testcontainerssession.SessionID())
}

type NoopImageSubstitutor struct{}

// Description returns a description of what is expected from this Substitutor,
// which is used in logs.
func (s NoopImageSubstitutor) Description() string {
    return "NoopImageSubstitutor (noop)"
}

// Substitute returns the original image, without any change
func (s NoopImageSubstitutor) Substitute(image string) (string, error) {
    return image, nil
}

Using the WithImageSubstitutors options, you could define your own substitutions to the container images. E.g. adding a prefix to the images so that they can be pulled from a Docker registry other than Docker Hub. This is the usual mechanism for using Docker image proxies, caches, etc.

Wait Strategies

If you need to set a different wait strategy for the container, you can use testcontainers.WithWaitStrategy with a valid wait strategy.

Info

The default deadline for the wait strategy is 60 seconds.

At the same time, it's possible to set a wait strategy and a custom deadline with testcontainers.WithWaitStrategyAndDeadline.

Startup Commands

Testcontainers exposes the WithStartupCommand(e ...Executable) option to run arbitrary commands in the container right after it's started.

Info

To better understand how this feature works, please read the Create containers: Lifecycle Hooks documentation.

It also exports an Executable interface, defining one single method: AsCommand(), which returns a slice of strings to represent the command and positional arguments to be executed in the container.

You could use this feature to run a custom script, or to run a command that is not supported by the module right after the container is started.

WithNetwork

  • Not available until the next release of testcontainers-go main

By default, the container is started in the default Docker network. If you want to use a different Docker network, you can use the WithNetwork(networkName string, alias string) option, which receives the new network name and an alias as parameters, creating the new network, attaching the container to it, and setting the network alias for that network.

If the network already exists, Testcontainers for Go won't create a new one, but it will attach the container to it and set the network alias.

Docker type modifiers

If you need an advanced configuration for the container, you can leverage the following Docker type modifiers:

  • testcontainers.WithConfigModifier
  • testcontainers.WithHostConfigModifier
  • testcontainers.WithEndpointSettingsModifier

Please read the Create containers: Advanced Settings documentation for more information.

Container Methods

The GCloud container exposes the following methods: