09 - List

This section implements the List method. List enables resource discovery - finding resources that exist in the infrastructure but aren't yet managed by formae.

What is Discovery?

Discovery allows formae to find "unmanaged" resources - resources that exist in your infrastructure but weren't created through formae. This is useful for:

• Importing existing infrastructure - Bring pre-existing resources under formae management
• Detecting drift - Find resources created outside of formae
• Inventory auditing - Get a complete view of what exists

The agent periodically runs discovery to scan for new resources. When it finds unmanaged resources, it can present them to the user for import.

┌─────────┐         ┌────────┐         ┌──────────────┐
│  Agent  │         │ Plugin │         │ Infrastructure│
└────┬────┘         └───┬────┘         └──────┬───────┘
     │                  │                     │
     │  List(File)      │                     │
     │─────────────────>│  List files         │
     │                  │────────────────────>│
     │  [file1, file2,  │  [file paths]       │
     │   file3]         │<────────────────────│
     │<─────────────────│                     │
     │                  │                     │
     │  [agent compares to inventory]         │
     │  [file3 is unmanaged - offer import]   │

List vs Read

• List returns resource identifiers (native IDs)
• Read is then called for each ID to get full properties

This two-phase approach allows efficient discovery - List is cheap (just IDs), and Read is only called when needed.

ListRequest

The agent sends a ListRequest during discovery:

type ListRequest struct {
    ResourceType         string            // Which resource type to discover
    TargetConfig         json.RawMessage   // Connection details for the target
    PageSize             int32             // Max results per page (0 = plugin default)
    PageToken            *string           // Resume token from previous page
    AdditionalProperties map[string]string // Plugin-specific discovery parameters
}

List returns a ListResult containing the native IDs found:

type ListResult struct {
    NativeIDs     []string // Infrastructure identifiers
    NextPageToken *string  // Set if more results exist
}

If your infrastructure has many resources, implement pagination by setting NextPageToken - the agent will call List again with this token until it's nil.

Test

Add a test for List to sftp_test.go:

// =============================================================================
// List Tests
// =============================================================================

// TestList verifies that the Plugin.List method:
// 1. Returns all file paths in the directory
// 2. Can be used for discovery of unmanaged resources
//
// Setup: Create multiple files using asyncsftp
// Execute: List using Plugin
// Verify: Assert returned paths include our files
func TestList(t *testing.T) {
    if os.Getenv("SFTP_USERNAME") == "" || os.Getenv("SFTP_PASSWORD") == "" {
        t.Skip("SFTP_USERNAME and SFTP_PASSWORD must be set")
    }

    ctx := context.Background()

    // --- Setup: Create files using asyncsftp ---
    client, err := asyncsftp.NewClient(testConfig())
    require.NoError(t, err, "failed to create asyncsftp client")
    defer client.Close()

    testFiles := []string{
        "/upload/test-list-1.txt",
        "/upload/test-list-2.txt",
        "/upload/test-list-3.txt",
    }

    for _, filePath := range testFiles {
        opID := client.StartUpload(filePath, "content", 0644)
        require.Eventually(t, func() bool {
            op, _ := client.GetStatus(opID)
            return op != nil && op.State == asyncsftp.StateCompleted
        }, 10*time.Second, 100*time.Millisecond, "file upload should complete")
    }

    // --- Execute: List using Plugin ---
    plugin := &Plugin{}

    req := &resource.ListRequest{
        ResourceType: "SFTP::Files::File",
        TargetConfig: testTargetConfig(),
    }

    result, err := plugin.List(ctx, req)
    require.NoError(t, err, "List should not return error")

    // --- Verify: Assert returned paths include our files ---
    for _, filePath := range testFiles {
        assert.Contains(t, result.NativeIDs, filePath,
            "List should include %s", filePath)
    }

    t.Logf("List returned %d files", len(result.NativeIDs))

    // --- Cleanup ---
    for _, filePath := range testFiles {
        _ = client.StartDelete(filePath)
    }
}

Test Pattern

Step	Action	Tool
Setup	Create multiple files	asyncsftp
Execute	List files	Plugin
Verify	Assert all files are returned	assert

Run the Test - Verify it Fails

SFTP_USERNAME=testuser SFTP_PASSWORD=testpass go test -tags=integration -run TestList -v

The test fails with not implemented.

Implementation

// List returns all resource identifiers of a given type.
// Called during discovery to find unmanaged resources.
func (p *Plugin) List(ctx context.Context, req *resource.ListRequest) (*resource.ListResult, error) {
    // Get SFTP client
    client, err := p.getClient(req.TargetConfig)
    if err != nil {
        return &resource.ListResult{
            NativeIDs: []string{},
        }, nil
    }

    // List files in the upload directory
    // In a real plugin, you might use AdditionalProperties to specify the directory
    dir := "/upload"
    if d, ok := req.AdditionalProperties["directory"]; ok {
        dir = d
    }

    paths, err := client.ListFiles(dir)
    if err != nil {
        // If directory doesn't exist, return empty list
        if errors.Is(err, asyncsftp.ErrNotFound) {
            return &resource.ListResult{
                NativeIDs: []string{},
            }, nil
        }
        return &resource.ListResult{
            NativeIDs: []string{},
        }, nil
    }

    return &resource.ListResult{
        NativeIDs:     paths,
        NextPageToken: nil, // No pagination for this simple implementation
    }, nil
}

Key Points

1. Return native IDs - Just the file paths, not full properties
2. Handle empty gracefully - Return empty list, not an error
3. Pagination support - Use NextPageToken for large result sets (not needed here)
4. AdditionalProperties - Can be used for discovery configuration (e.g., which directory to scan)

Verify

Run the test to confirm List discovers files correctly:

SFTP_USERNAME=testuser SFTP_PASSWORD=testpass go test -tags=integration -run TestList -v

The test creates several files, then verifies that List returns all of them:

=== RUN   TestList
    sftp_test.go:482: List returned 6 files
--- PASS: TestList (0.72s)
PASS

Discovery Flow

Here's how discovery works end-to-end:

1. Agent calls List → Plugin returns all native IDs
2. Agent compares to inventory → Identifies unmanaged resources
3. Agent calls Read for each unmanaged resource → Gets full properties
4. Agent presents to user → User can import into formae management

Non-Discoverable Resources

Some resource types cannot be listed - for example, if the infrastructure API doesn't provide a list endpoint. In these cases, mark the resource as non-discoverable in its schema:

@formae.ResourceHint {
    type = "Example::Service::Resource"
    identifier = "$.id"
    discoverable = false
}

The agent will skip these resource types during discovery scans. Resources are discoverable by default, so you only need to set this when explicitly disabling discovery.

Summary

List enables resource discovery:

Aspect	Description
Purpose	Find unmanaged resources
Returns	Native IDs only (not full properties)
Pagination	Use NextPageToken for large result sets
Errors	Return empty list, not errors

With List implemented, your plugin supports the complete discovery workflow.

---

Next: 10 - Error Handling - Deep dive into OperationErrorCode patterns