Overview

The srclib API will be used through the invocation of subcommands of the srclib executable.

API Commands

API commands return their responses as JSON, to facilitate the building of tools on top of srclib. Sourcegraph's plugins all make heavy use of the API commands.

srclib api describe

This command is used by editor plugins to retrieve information about
the identifier at a specific position in a file.

It will hit Sourcegraph's API to get a definition's examples. With the
flag `--no-examples`, this command does not hit Sourcegraph's API.

Usage

$ srclib api describe -h
Usage:
  srclib [OPTIONS] api describe [describe-OPTIONS]

Returns information about the definition referred to by the cursor's current position in a file.

Global options:
  -v                           show verbose output

Help Options:
  -h, --help                   Show this help message

[describe command options]
          --file=FILE
          --start-byte=BYTE

Output

The output is defined in api_cmds.go, as the JSON representation of the following struct.

The Def and Example structs are defined as follows in the Sourcegraph API.

type apiDescribeCmdOutput struct {
    Def *graph.Def
}

// Def is a definition in code.
type Def struct {
    // DefKey is the natural unique key for a def. It is stable
    // (subsequent runs of a grapher will emit the same defs with the same
    // DefKeys).
    DefKey `protobuf:"bytes,1,opt,name=key,embedded=key" json:""`
    // Name of the definition. This need not be unique.
    Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"Name"`
    // Kind is the kind of thing this definition is. This is
    // language-specific. Possible values include "type", "func",
    // "var", etc.
    Kind     string `protobuf:"bytes,3,opt,name=kind,proto3" json:"Kind,omitempty"`
    File     string `protobuf:"bytes,4,opt,name=file,proto3" json:"File"`
    DefStart uint32 `protobuf:"varint,5,opt,name=start,proto3" json:"DefStart"`
    DefEnd   uint32 `protobuf:"varint,6,opt,name=end,proto3" json:"DefEnd"`
    // Exported is whether this def is part of a source unit's
    // public API. For example, in Java a "public" field is
    // Exported.
    Exported bool `protobuf:"varint,7,opt,name=exported,proto3" json:"Exported,omitempty"`
    // Local is whether this def is local to a function or some
    // other inner scope. Local defs do *not* have module,
    // package, or file scope. For example, in Java a function's
    // args are Local, but fields with "private" scope are not
    // Local.
    Local bool `protobuf:"varint,8,opt,name=local,proto3" json:"Local,omitempty"`
    // Test is whether this def is defined in test code (as opposed to main
    // code). For example, definitions in Go *_test.go files have Test = true.
    Test bool `protobuf:"varint,9,opt,name=test,proto3" json:"Test,omitempty"`
    // Data contains additional language- and toolchain-specific information
    // about the def. Data is used to construct function signatures,
    // import/require statements, language-specific type descriptions, etc.
    Data sourcegraph_com_sqs_pbtypes.RawMessage `protobuf:"bytes,10,opt,name=data,proto3,customtype=sourcegraph.com/sqs/pbtypes.RawMessage" json:"Data,omitempty"`
    // Docs are docstrings for this Def. This field is not set in the
    // Defs produced by graphers; they should emit docs in the
    // separate Docs field on the graph.Output struct.
    Docs []*DefDoc `protobuf:"bytes,11,rep,name=docs" json:"Docs,omitempty"`
    // TreePath is a structurally significant path descriptor for a def. For
    // many languages, it may be identical or similar to DefKey.Path.
    // However, it has the following constraints, which allow it to define a
    // def tree.
    //
    // A tree-path is a chain of '/'-delimited components. A component is either a
    // def name or a ghost component.
    // - A def name satifies the regex [^/-][^/]*
    // - A ghost component satisfies the regex -[^/]*
    // Any prefix of a tree-path that terminates in a def name must be a valid
    // tree-path for some def.
    // The following regex captures the children of a tree-path X: X(/-[^/]*)*(/[^/-][^/]*)
    TreePath string `protobuf:"bytes,17,opt,name=tree_path,proto3" json:"TreePath,omitempty"`
}

// Ref represents a reference from source code to a Def.
type Ref struct {
    // DefRepo is the repository URI of the Def that this Ref refers
    // to.
    DefRepo string `protobuf:"bytes,1,opt,name=def_repo,proto3" json:"DefRepo,omitempty"`
    // DefUnitType is the source unit type of the Def that this Ref refers to.
    DefUnitType string `protobuf:"bytes,3,opt,name=def_unit_type,proto3" json:"DefUnitType,omitempty"`
    // DefUnit is the name of the source unit that this ref exists in.
    DefUnit string `protobuf:"bytes,4,opt,name=def_unit,proto3" json:"DefUnit,omitempty"`
    // Path is the path of the Def that this ref refers to.
    DefPath string `protobuf:"bytes,5,opt,name=def_path,proto3" json:"DefPath"`
    // Repo is the VCS repository in which this ref exists.
    Repo string `protobuf:"bytes,6,opt,name=repo,proto3" json:"Repo,omitempty"`
    // CommitID is the ID of the VCS commit that this ref exists
    // in. The CommitID is always a full commit ID (40 hexadecimal
    // characters for git and hg), never a branch or tag name.
    CommitID string `protobuf:"bytes,7,opt,name=commit_id,proto3" json:"CommitID,omitempty"`
    // UnitType is the type name of the source unit that this ref
    // exists in.
    UnitType string `protobuf:"bytes,8,opt,name=unit_type,proto3" json:"UnitType,omitempty"`
    // Unit is the name of the source unit that this ref exists in.
    Unit string `protobuf:"bytes,9,opt,name=unit,proto3" json:"Unit,omitempty"`
    // Def is true if this Ref spans the name of the Def it points to.
    Def bool `protobuf:"varint,17,opt,name=def,proto3" json:"Def,omitempty"`
    // File is the filename in which this Ref exists.
    File string `protobuf:"bytes,10,opt,name=file,proto3" json:"File,omitempty"`
    // Start is the byte offset of this ref's first byte in File.
    Start uint32 `protobuf:"varint,11,opt,name=start,proto3" json:"Start"`
    // End is the byte offset of this ref's last byte in File.
    End uint32 `protobuf:"varint,12,opt,name=end,proto3" json:"End"`
}

srclib api list

This command will return a list of all the definitions,
references, and docs in a file. It can be used for finding all
uses of a reference in a file.

Usage

$ srclib api list -h
Usage:
  srclib [OPTIONS] api list [list-OPTIONS]

Return a list of all definitions, references, and docs that are in the current file.

Global options:
  -v                     show verbose output

Help Options:
  -h, --help             Show this help message

[list command options]
          --file=FILE
          --no-refs
          --no-defs
          --no-docs

Output

type apiListCmdOutput struct { Defs []graph.Def json:",omitempty" Refs []graph.Ref json:",omitempty" Docs []*graph.Doc json:",omitempty" }

srclib api deps

This command returns a list of all resolved and unresolved
dependencies for the current repository.

Usage

$ srclib api list -h
Usage:
  srclib [OPTIONS] api list [list-OPTIONS]

Return a list of all definitions, references, and docs that are in the current file.

Global options:
  -v                     show verbose output

Help Options:
  -h, --help             Show this help message

[list command options]
          --file=FILE
          --no-refs
          --no-defs
          --no-docs

Output

This command returns a dep.Resolution slice.

// Resolution is the result of dependency resolution: either a successfully
// resolved target or an error.
type Resolution struct {
    // Raw is the original raw dep that this was resolution was attempted on.
    Raw interface{}

    // Target is the resolved dependency, if resolution succeeds.
    Target *ResolvedTarget `json:",omitempty"`

    // Error is the resolution error, if any.
    Error string `json:",omitempty"`
}

srclib api units

This command returns a list of all of the source units in the current
repository.

Usage

$ srclib api units -h
Usage:
  srclib [OPTIONS] api units [Dir]

Return a list of all source units that are in the current repository.

Global options:
  -v              show verbose output

Help Options:
  -h, --help      Show this help message

[units command arguments]
  Dir:            root directory of target project

Output

This command returns a unit.SourceUnit slice.

type SourceUnit struct {
    // Name is an opaque identifier for this source unit that MUST be unique
    // among all other source units of the same type in the same repository.
    //
    // Two source units of different types in a repository may have the same name.
    // To obtain an identifier for a source unit that is guaranteed to be unique
    // repository-wide, use the ID method.
    Name string

    // Type is the type of source unit this represents, such as "GoPackage".
    Type string

    // Repo is the URI of the repository containing this source unit, if any.
    // The scanner tool does not need to set this field - it can be left blank,
    // to be filled in by the `srclib` tool
    Repo string `json:",omitempty"`

    // CommitID is the commit ID of the repository containing this
    // source unit, if any. The scanner tool need not fill this in; it
    // should be left blank, to be filled in by the `srclib` tool.
    CommitID string `json:",omitempty"`

    // Globs is a list of patterns that match files that make up this source
    // unit. It is used to detect when the source unit definition is out of date
    // (e.g., when a file matches the glob but is not in the Files list).
    //
    // TODO(sqs): implement this in the Makefiles
    Globs []string `json:",omitempty"`

    // Files is all of the files that make up this source unit. Filepaths should
    // be relative to the repository root.
    Files []string

    // Dir is the root directory of this source unit. It is optional and maybe
    // empty.
    Dir string `json:",omitempty"`

    // Dependencies is a list of dependencies that this source unit has. The
    // schema for these dependencies is internal to the scanner that produced
    // this source unit. The dependency resolver is expected to know how to
    // interpret this schema.
    //
    // The dependency information stored in this field should be able to be very
    // quickly determined by the scanner. The scanner should not perform any
    // dependency resolution on these entries. This is because the scanner is
    // run frequently and should execute very quickly, and dependency resolution
    // is often slow (requiring network access, etc.).
    Dependencies []interface{} `json:",omitempty"`

    // Info is an optional field that contains additional information used to
    // display the source unit
    Info *Info `json:",omitempty"`

    // Data is additional data dumped by the scanner about this source unit. It
    // typically holds information that the scanner wants to make available to
    // other components in the toolchain (grapher, dep resolver, etc.).
    Data interface{} `json:",omitempty"`

    // Config is an arbitrary key-value property map. The Config map from the
    // tree config is copied verbatim to each source unit. It can be used to
    // pass options from the Srcfile to tools.
    Config map[string]interface{} `json:",omitempty"`

    // Ops enumerates the operations that should be performed on this source
    // unit. Each key is the name of an operation, and the value is the tool to
    // use to perform that operation. If the value is nil, the tool is chosen
    // automatically according to the user's configuration.
    Ops map[string]*srclib.ToolRef `json:",omitempty"`

    // TODO(sqs): add a way to specify the toolchains and tools to use for
    // various tasks on this source unit
}

func (u *SourceUnit) Key() Key {
    return Key{
        Repo:     u.Repo,
        CommitID: u.CommitID,
        UnitType: u.Type,
        Unit:     u.Name,
    }
}

Standalone Commands

Standalong commands are for the srclib power user: most people will use srclib through an editor plugin or Sourcegraph, but the following commands are useful for modifying the state of a repository's analysis data.

srclib config

srclib config is used to detect what kinds of source units (npm/pip/Go/etc. packages) exist in a repository or directory tree.

srclib make

srclib make is used to perform analysis on a given directory. See the srclib make docs for usage instructions.