TestPicker
Documentation for TestPicker.
TestPicker.INTERFACES
TestPicker.LATEST_EVAL
TestPicker.TESTENV_CACHE
TestPicker.TESTMODE_TRIGGER
TestPicker.EvalTest
TestPicker.QueryType
TestPicker.StdTestset
TestPicker.SyntaxBlock
TestPicker.TestBlockInfo
TestPicker.TestBlockInterface
TestPicker.TestInfo
TestPicker.add_interface!
TestPicker.blocklabel
TestPicker.build_info_to_syntax
TestPicker.clean_results_file
TestPicker.clean_source
TestPicker.clear_testenv_cache
TestPicker.create_repl_test_mode
TestPicker.current_pkg
TestPicker.eval_in_module
TestPicker.expr_transform
TestPicker.fzf_testblock
TestPicker.fzf_testfile
TestPicker.get_matching_files
TestPicker.get_preview_dimension
TestPicker.get_test_files
TestPicker.get_testblocks
TestPicker.identify_query
TestPicker.init_test_repl_mode
TestPicker.ispreamble
TestPicker.istestblock
TestPicker.pick_testblock
TestPicker.preamble
TestPicker.prepend_ex
TestPicker.prepend_preamble_statements
TestPicker.preview_content
TestPicker.remove_ansi
TestPicker.replace_interface!
TestPicker.run_test_file
TestPicker.run_test_files
TestPicker.save_test_results
TestPicker.select_test_files
TestPicker.separator
TestPicker.test_mode_do_cmd
TestPicker.testblock_list
TestPicker.visualize_test_results
TestPicker.INTERFACES
— ConstantINTERFACES
Global collection of test block interfaces used by TestPicker.
Contains all registered TestBlockInterface
implementations that TestPicker uses to recognize and parse different types of test blocks. By default includes StdTestset
for standard @testset
blocks.
Type
Vector{TestBlockInterface}
: Collection of interface implementations
Default Contents
StdTestset
: Handles standard Julia@testset
blocks
Modification
Use add_interface!
to register additional test block interfaces:
# Add support for custom test blocks
add_interface!(MyCustomTestInterface())
TestPicker.LATEST_EVAL
— ConstantLATEST_EVAL
Global reference to the most recently executed test evaluations.
Stores a vector of EvalTest
objects representing the last set of tests that were executed. This allows for re-running the same tests without going through the selection interface again.
Type
Ref{Union{Nothing,Vector{EvalTest}}}
: Initially nothing
, becomes a vector after first test execution
Usage
- Set automatically when tests are executed
- Can be re-run using the
-
command in test mode - Cleared when new test files are executed (not individual test blocks)
Examples
# In test mode REPL:
# Run some tests first, then:
test> - # Re-runs the last executed tests
TestPicker.TESTENV_CACHE
— ConstantTESTENV_CACHE
Cache for TestEnv temporary environments to avoid triggering recompilation on every test run.
This constant stores a mapping from PackageSpec
objects to their corresponding temporary test environment paths. Reusing these environments improves performance by avoiding the overhead of recreating test environments.
TestPicker.TESTMODE_TRIGGER
— ConstantTrigger key to get into test mode.
TestPicker.EvalTest
— TypeEvalTest
Container for executable test code and its associated metadata.
Combines a Julia expression representing test code with metadata about its source and context. Used throughout TestPicker for tracking and executing tests.
Fields
ex::Expr
: The executable test expression (may include preamble and wrapper code)info::TestInfo
: Metadata about the test's source and identification
Usage
EvalTest
objects are created during test parsing and stored in LATEST_EVAL
for re-execution and result tracking.
TestPicker.QueryType
— TypeQueryType
Enumeration of different types of test queries supported by the test REPL mode.
Values
TestFileQuery
: Query for running tests from specific filesTestsetQuery
: Query for running specific test sets (format: "file:testset")LatestEval
: Re-run the most recently evaluated tests (triggered by "-")InspectResults
: View test results visualization (triggered by "?")UnmatchedQuery
: Query that couldn't be parsed or is invalid
TestPicker.StdTestset
— TypeStdTestset <: TestBlockInterface
Standard implementation of TestBlockInterface
for @testset
blocks from Julia's Test.jl
standard library.
This is the built-in interface for recognizing and processing standard Julia test sets. It handles @testset
blocks commonly used in Julia testing and provides the necessary preamble to load the Test.jl package.
Behavior
- Recognizes syntax nodes that start with
@testset
- Extracts test set names as labels for display
- Automatically includes
using Test
as preamble - No expression transformation (uses default identity)
Examples
# This testset would be recognized:
@testset "Basic arithmetic tests" begin
@test 1 + 1 == 2
@test 2 * 3 == 6
end
# The label would be: "Basic arithmetic tests"
TestPicker.SyntaxBlock
— TypeSyntaxBlock
A container for a test block and its associated preamble statements.
Contains all the necessary components to execute a test block, including any setup code that needs to run beforehand. Can be easily converted into an evaluatable expression.
Fields
preamble::Vector{SyntaxNode}
: Collection of preamble statements (imports, assignments, etc.)testblock::SyntaxNode
: The actual test block node (e.g.,@testset
)interface::TestBlockInterface
: The interface implementation used to parse this block
TestPicker.TestBlockInfo
— TypeTestBlockInfo
Metadata container for a test block, including its location and identification information.
Stores essential information about a test block's location within a file and provides a label for identification and display purposes.
Fields
label::String
: Human-readable label for the test block (e.g., test set name)file_name::String
: Name of the file containing the test blockline_start::Int
: Starting line number of the test block (1-indexed)line_end::Int
: Ending line number of the test block (1-indexed)
TestPicker.TestBlockInterface
— TypeTestBlockInterface
Abstract interface for defining and recognizing different types of test blocks in Julia code.
The TestBlockInterface
allows you to define different types of test blocks that you would like TestPicker
to find and evaluate. The interface is relatively simple and flexible.
Implementation Requirements
To create a custom test block interface, define a subtype and implement the required methods:
struct MyTestBlock <: TestBlockInterface end
Required Methods
istestblock(::MyTestBlock, node::SyntaxNode)::Bool
: Determines whether a givenSyntaxNode
represents a test blockblocklabel(::MyTestBlock, node::SyntaxNode)::String
: Produces a (preferably) unique label for filtering and display
Optional Methods
preamble(::MyTestBlock)::Union{Nothing, Expr}
: Returns additional preamble that the test block might require (default:nothing
)expr_transform(::MyTestBlock, ex::Expr)::Expr
: Transforms the test block expression before evaluation (default: identity)
Examples
# Define a custom interface for @test_nowarn blocks
struct NoWarnTestInterface <: TestBlockInterface end
function istestblock(::NoWarnTestInterface, node::SyntaxNode)
nodes = JuliaSyntax.children(node)
return !isnothing(nodes) && !isempty(nodes) &&
Expr(first(nodes)) == Symbol("@test_nowarn")
end
function blocklabel(::NoWarnTestInterface, node::SyntaxNode)
return "nowarn: " * JuliaSyntax.sourcetext(JuliaSyntax.children(node)[2])
end
TestPicker.TestInfo
— TypeTestInfo
Container for test execution metadata and location information.
Stores essential information about a test's source location and context, used for tracking test execution and displaying results.
Fields
filename::String
: Source file containing the testlabel::String
: Human-readable test identifier (e.g., testset name)line::Int
: Line number where the test begins
TestPicker.add_interface!
— Methodadd_interface!(interface::TestBlockInterface) -> Vector{TestBlockInterface}
Register a new test block interface with TestPicker.
Adds the provided interface to the global INTERFACES
collection, enabling TestPicker to recognize and process the corresponding test block types. Duplicates are automatically removed to prevent redundant processing.
Arguments
interface::TestBlockInterface
: The interface implementation to register
Returns
Vector{TestBlockInterface}
: The updated interfaces collection
Examples
# Define a custom interface
struct MyTestInterface <: TestBlockInterface end
# ... implement required methods ...
# Register with TestPicker
add_interface!(MyTestInterface())
# Now TestPicker will recognize your test blocks
TestPicker.blocklabel
— Methodblocklabel(interface::T, node::SyntaxNode)::String where {T<:TestBlockInterface}
Generate a descriptive label for a test block to be used in filtering and display.
This is a required method that must be implemented by all concrete subtypes of TestBlockInterface
. It should produce a (preferably) unique label that helps users identify and select specific test blocks.
Arguments
interface::T
: The test block interface implementationnode::SyntaxNode
: The syntax node representing the test block
Returns
String
: A descriptive label for the test block
Implementation Notes
- Labels should be human-readable and distinctive
- Consider including test names, descriptions, or other identifying information
Examples
# For @testset blocks, extract the test set name:
function blocklabel(::StdTestset, node::SyntaxNode)
return JuliaSyntax.sourcetext(JuliaSyntax.children(node)[2])
end
TestPicker.build_info_to_syntax
— Methodbuild_info_to_syntax(interfaces, root, matched_files) -> (Dict{TestBlockInfo,SyntaxBlock}, Dict{String,TestBlockInfo})
Parse matched files and build mapping structures for test block selection and display.
Extracts all test blocks from the provided files and creates two mappings:
- From test block metadata to syntax information
- From human-readable display strings (for fzf) to test block metadata
Arguments
interfaces::Vector{<:TestBlockInterface}
: Test block interfaces for parsingroot::AbstractString
: Root directory path for resolving relative file pathsmatched_files::AbstractVector{<:AbstractString}
: List of files to parse
Returns
Tuple{Dict{TestBlockInfo,SyntaxBlock}, Dict{String,TestBlockInfo}}
:- First element: Maps test block info to syntax blocks
- Second element: Maps formatted display strings to test block info
Notes
The display strings are formatted with padding for alignment in fzf, showing: "<label> | <filename>:<line_start>-<line_end>"
TestPicker.clean_results_file
— MethodThis empty the file before appending new results.
TestPicker.clean_source
— MethodObtain the source from the LineNumberNode.
TestPicker.clear_testenv_cache
— Methodclear_testenv_cache()
Clear the TestEnv cache to force recreation of test environments on next use.
Empties the TESTENV_CACHE
dictionary, which will cause subsequent test evaluations to create fresh test environments. This can be useful when test dependencies have changed or when troubleshooting environment-related issues.
TestPicker.create_repl_test_mode
— Methodcreate_repl_test_mode(repl::AbstractREPL, main::LineEdit.Prompt) -> LineEdit.Prompt
Create a new REPL mode specifically for test operations.
Constructs a custom REPL prompt mode that handles test-specific commands and provides an isolated interface for TestPicker operations. The mode includes proper history support, key bindings, and command processing.
Arguments
repl::AbstractREPL
: The REPL instance to create the mode formain::LineEdit.Prompt
: The main REPL mode to inherit settings from
Returns
LineEdit.Prompt
: The configured test mode prompt ready for use
Features
- Custom prompt with magenta coloring (if supported)
- Sticky mode behavior for continued test operations
- Integrated history and search functionality
- Error handling for test execution failures
- Automatic return to main mode when appropriate
TestPicker.current_pkg
— Methodcurrent_pkg() -> PackageSpec
Get the current package specification from the active Pkg environment.
This is a more flexible version of current_pkg_name
from TestEnv
that returns the full PackageSpec
object rather than just the package name. This provides access to additional package metadata needed by TestPicker.
Returns
PackageSpec
: The package specification for the current project
Throws
TestEnvError
: If trying to activate test environment of an unnamed project
Notes
The function examines the current Pkg context to determine the active package. This is essential for TestPicker to understand which package's tests are being executed and where to store results.
TestPicker.eval_in_module
— Methodeval_in_module(eval_test::EvalTest, pkg::PackageSpec) -> Nothing
Execute a test block in an isolated module with the appropriate test environment activated.
This function provides the core test execution functionality for TestPicker. It creates a temporary module, activates the package's test environment, and evaluates the test code in isolation to prevent interference between different test runs.
Arguments
eval_test::EvalTest
: Contains the test expression and metadatapkg::PackageSpec
: Package specification for environment setup
Process Overview
- Environment Setup: Activates test environment (cached or new)
- Module Creation: Creates isolated module for test execution
- Code Preparation: Prepends necessary imports and setup code
- Execution: Evaluates test in correct directory context with Revise support
- Cleanup: Restores original environment and cleans temporary module
Side Effects
- Temporarily changes active Pkg environment
- Creates and cleans up temporary module in Main
- May trigger Revise.revise() for code reloading
- Changes working directory during execution
- Outputs execution information via
@info
Environment Management
- Uses
TESTENV_CACHE
for performance optimization - Temporarily disables auto-precompilation during setup
- Restores original environment and precompilation settings after execution
Error Handling
- Ensures environment restoration even if test execution fails
- Cleans up temporary module variables in finally block
- Preserves original working directory
Examples
# Execute a test block from an EvalTest object
test = EvalTest(:(using Test; @test 1+1 == 2), TestInfo("test.jl", "arithmetic", 1))
pkg = PackageSpec(name="MyPackage", path="/path/to/package")
eval_in_module(test, pkg)
TestPicker.expr_transform
— Methodexpr_transform(interface::TestBlockInterface, ex::Expr)::Expr
Transform a test block expression before evaluation.
This optional method allows test block interfaces to modify the test block expression before it is executed. This can be useful for adding wrapper code, modifying test behavior, or adapting different test formats.
Arguments
interface::TestBlockInterface
: The test block interface implementationex::Expr
: The test block expression to transform
Returns
Expr
: The transformed expression ready for evaluation
Default Behavior
The default implementation returns the expression unchanged (identity transformation).
Examples
# Add timing information to test blocks:
function expr_transform(::TimedTestInterface, ex::Expr)
return quote
start_time = time()
result = $ex
elapsed = time() - start_time
println("Test completed in $(elapsed)s")
result
end
end
# Wrap tests in additional error handling:
function expr_transform(::SafeTestInterface, ex::Expr)
return quote
try
$ex
catch e
@warn "Test failed with error: $e"
rethrow()
end
end
end
TestPicker.fzf_testblock
— Methodfzf_testblock(interfaces, fuzzy_file, fuzzy_testset) -> Nothing
Interactive test block selection and execution workflow using fzf.
Provides a two-stage fuzzy finding process:
- Filter test files based on
fuzzy_file
query - Select specific test blocks from filtered files based on
fuzzy_testset
query
Selected test blocks are executed immediately with results saved to the package's results file. The latest evaluation is stored in LATEST_EVAL[]
for reference.
Arguments
interfaces::Vector{<:TestBlockInterface}
: Test block interfaces for parsingfuzzy_file::AbstractString
: Query pattern for filtering test filesfuzzy_testset::AbstractString
: Query pattern for filtering test block names
Side Effects
- Executes selected test blocks
- Updates
LATEST_EVAL[]
with executed tests - Cleans and writes to results file
- May modify package test state
Examples
# Find and run tests matching "math" files and "addition" test names
fzf_testblock([StdTestset()], "math", "addition")
TestPicker.fzf_testfile
— Methodfzf_testfile(query::AbstractString) -> Nothing
Interactive test file selection and execution workflow.
Combines file selection and execution in a single workflow: uses fzf to select test files based on the query, then runs all selected files in the test environment.
Arguments
query::AbstractString
: Initial search pattern for filtering test files
Process
- Get current package context
- Present fzf interface for file selection
- Execute all selected test files
- Handle results and error reporting
Side Effects
- Updates
LATEST_EVAL[]
with executed tests - Clears and populates results file with any failures/errors
- May modify test environment state
Examples
# Run tests matching "integration"
fzf_testfile("integration")
# Run all test files (empty query shows all)
fzf_testfile("")
TestPicker.get_matching_files
— Methodget_matching_files(file_query::AbstractString, test_files::AbstractVector{<:AbstractString}) -> Vector{String}
Filter test files using fzf's non-interactive filtering based on the given query.
Uses fzf --filter
to perform fuzzy matching on the provided list of test files, returning only those that match the query pattern.
Arguments
file_query::AbstractString
: Fuzzy search pattern to match against file namestest_files::AbstractVector{<:AbstractString}
: List of test file paths to filter
Returns
Vector{String}
: List of file paths that match the query
Examples
files = ["test/test_math.jl", "test/test_string.jl", "test/integration.jl"]
get_matching_files("math", files) # Returns ["test/test_math.jl"]
TestPicker.get_preview_dimension
— FunctionUtility function to adapt the size of the text width and line position.
TestPicker.get_test_files
— Functionget_test_files(pkg::PackageSpec=current_pkg()) -> (String, Vector{String})
Discover and return all Julia test files for a package.
Recursively searches the package's test directory to find all .jl
files, returning both the test directory path and the collection of relative file paths.
Arguments
pkg::PackageSpec
: Package specification (defaults to current package)
Returns
Tuple{String, Vector{String}}
:- First element: Absolute path to the test directory
- Second element: Vector of test file paths relative to test directory
Notes
- Only includes files with
.jl
extension - Searches recursively through subdirectories
- File paths are normalized and relative to test directory root
Examples
test_dir, files = get_test_files()
# test_dir: "/path/to/MyPackage/test"
# files: ["runtests.jl", "subdir/test_math.jl", "test_utils.jl"]
TestPicker.get_testblocks
— Methodget_testblocks(interfaces::Vector{<:TestBlockInterface}, file::AbstractString) -> Vector{SyntaxBlock}
Parse a Julia file and extract all test blocks with their associated preamble statements.
For each test block found (including nested ones), collects all preceding preamble statements that should be executed before the test block. Uses the provided interfaces to determine what constitutes a test block.
Arguments
interfaces::Vector{<:TestBlockInterface}
: Collection of test block interfaces to use for parsingfile::AbstractString
: Path to the Julia file to parse
Returns
Vector{SyntaxBlock}
: Collection of parsed test blocks with their preambles
TestPicker.identify_query
— Methodidentify_query(input::AbstractString) -> (QueryType, Tuple)
Parse user input in test mode and identify the type of operation requested.
Analyzes the input string to determine what kind of test operation the user wants to perform and extracts the relevant parameters for that operation.
Arguments
input::AbstractString
: The raw input from the test mode REPL
Returns
Tuple{QueryType, Tuple}
: The query type and associated parameters
Query Formats
"-"
: Re-run latest evaluation (returnsLatestEval
with stored tests)"?"
: Inspect test results (returnsInspectResults
with empty tuple)"file:testset"
: Run specific test set (returnsTestsetQuery
with file and testset)"filename"
: Run tests from file (returnsTestFileQuery
with filename and empty testset)
Examples
identify_query("test_math.jl") # (TestFileQuery, ("test_math.jl", ""))
identify_query("test_math.jl:addition") # (TestsetQuery, ("test_math.jl", "addition"))
identify_query("-") # (LatestEval, <previous_tests>)
identify_query("?") # (InspectResults, ())
Error Handling
- Returns
UnmatchedQuery
for inputs that cannot be parsed - Handles case where no previous evaluation exists for "-" command
TestPicker.init_test_repl_mode
— Methodinit_test_repl_mode(repl::AbstractREPL) -> Nothing
Initialize and add test mode to the REPL interface.
Sets up a custom REPL mode for TestPicker that can be accessed by typing '!' at the beginning of a line. The test mode provides specialized commands for running and inspecting tests interactively.
Arguments
repl::AbstractREPL
: The REPL instance to modify
Side Effects
- Adds a new test mode to the REPL interface
- Modifies the main mode's keymap to include the trigger
- Sets up mode switching behavior and key bindings
Notes
The test mode supports:
- Standard REPL features (history, search, etc.)
- Custom test commands and query parsing
- Seamless switching between main and test modes
TestPicker.ispreamble
— Methodispreamble(node::SyntaxNode) -> Bool
Check if a statement qualifies as a preamble that should be executed before test blocks.
A preamble statement is any statement that sets up the testing environment, such as:
- Function calls (
:call
) - Import/using statements (
:using
,:import
) - Variable assignments (
:=
) - Macro calls (
:macrocall
) - Function definitions (
:function
)
Arguments
node::SyntaxNode
: The syntax node to check
Returns
Bool
:true
if the node represents a preamble statement,false
otherwise
Examples
# These would return true:
using Test
import MyModule
x = 10
@testset "setup" begin end
function helper() end
TestPicker.istestblock
— Methodistestblock(interface::T, node::SyntaxNode)::Bool where {T<:TestBlockInterface}
Determine whether a syntax node represents a test block according to the given interface.
This is a required method that must be implemented by all concrete subtypes of TestBlockInterface
. It examines a syntax node and decides whether it represents a test block that should be recognized by TestPicker.
Arguments
interface::T
: The test block interface implementationnode::SyntaxNode
: The syntax node to examine
Returns
Bool
:true
if the node represents a test block,false
otherwise
Examples
# For @testset blocks:
function istestblock(::StdTestset, node::SyntaxNode)
nodes = JuliaSyntax.children(node)
return !isnothing(nodes) && !isempty(nodes) &&
Expr(first(nodes)) == Symbol("@testset")
end
TestPicker.pick_testblock
— Methodpick_testblock(tabled_keys, testset_query, root) -> Vector{String}
Present an interactive fzf interface for selecting test blocks to execute.
Launches fzf with a preview window (using bat) that allows users to select one or more test blocks from the filtered list. The preview shows the actual test code with syntax highlighting.
Arguments
tabled_keys::Dict{String,TestBlockInfo}
: Mapping from display strings to test block infotestset_query::AbstractString
: Initial query to filter test block namesroot::AbstractString
: Root directory for file path resolution
Returns
Vector{String}
: List of selected display strings corresponding to chosen test blocks
Features
- Multi-selection enabled (
-m
flag) - Preview window shows test code with syntax highlighting
- Initial query pre-filters results
- Search limited to visible test labels (not file paths)
TestPicker.preamble
— Methodpreamble(interface::TestBlockInterface)::Union{Nothing, Expr}
Return additional preamble code specific to the test block interface.
This optional method allows test block interfaces to specify setup code that should be executed before any test blocks of this type. Common uses include importing required packages or setting up test environment variables.
Arguments
interface::TestBlockInterface
: The test block interface implementation
Returns
Union{Nothing, Expr}
: Preamble expression to execute, ornothing
if no preamble needed
Default Behavior
The default implementation returns nothing
, indicating no additional preamble is required.
Examples
# StdTestset requires Test.jl to be loaded:
function preamble(::StdTestset)
return :(using Test)
end
# Custom interface might need multiple setup steps:
function preamble(::MyCustomInterface)
return quote
using Test, Random
Random.seed!(1234)
end
end
TestPicker.prepend_ex
— Methodprepend_ex(ex, new_line::Expr) -> Expr
Prepend a new expression to an existing expression, handling block structure appropriately.
If the target expression is already a block, the new expression is prepended to the beginning of the block. Otherwise, a new block is created containing both expressions.
TestPicker.prepend_preamble_statements
— Methodprepend_preamble_statements(interface::TestBlockInterface, preambles::Vector{Expr}) -> Vector{Expr}
Combine interface-specific preamble with existing preamble statements.
Takes the preamble from the interface (if any) and prepends it to the existing collection of preamble statements, ensuring interface requirements are satisfied before test execution.
Arguments
interface::TestBlockInterface
: The test block interface implementationpreambles::Vector{Expr}
: Existing preamble statements from the test file
Returns
Vector{Expr}
: Combined preamble statements with interface preamble first
Examples
interface = StdTestset()
existing = [:(x = 1), :(y = 2)]
result = prepend_preamble_statements(interface, existing)
# Returns: [:(using Test), :(x = 1), :(y = 2)]
TestPicker.preview_content
— MethodWe connect the error with the backtrace to be previewed.
TestPicker.remove_ansi
— MethodFile names come with ansi characters and break stuff...
TestPicker.replace_interface!
— Methodreplace_interface!(interface::TestBlockInterface) -> Vector{TestBlockInterface}
Similar to add_interface!
but empty the interface first before adding the new one so that it becomes the unique interface.
TestPicker.run_test_file
— Methodrun_test_file(file::AbstractString, pkg::PackageSpec) -> Any
Execute a single test file in an isolated testset within the package test environment.
Wraps the test file in a testset named after the package and file, handles test failures gracefully, and updates the global test state for later inspection.
Arguments
file::AbstractString
: Path to the test file to executepkg::PackageSpec
: Package specification for environment and naming
Process
- Creates a testset named "{package} - {file}"
- Includes the test file within the testset
- Catches and saves any test failures/errors
- Updates
LATEST_EVAL[]
with the test execution - Evaluates in isolated module environment
Test Structure
The file is executed within this structure:
@testset "PackageName - filepath" begin
include("filepath")
end
Side Effects
- Updates or initializes
LATEST_EVAL[]
global state - May save test results to results file on failures
- Executes file in test environment context
Error Handling
- Test failures are caught and saved rather than propagated
- Non-test errors are re-thrown
- File-level errors are properly contextualized
TestPicker.run_test_files
— Methodrun_test_files(files::AbstractVector{<:AbstractString}, pkg::PackageSpec) -> Nothing
Execute a collection of test files in the package test environment.
Runs each provided test file in sequence, handling errors gracefully and updating the test evaluation state. Each file is wrapped in a testset and executed in isolation.
Arguments
files::AbstractVector{<:AbstractString}
: Full paths to test files to executepkg::PackageSpec
: Package specification for test environment context
Behavior
- Returns early if no files provided (preserves existing
LATEST_EVAL
state) - Resets
LATEST_EVAL[]
to empty array for new test run - Clears results file before execution
- Validates file existence before attempting execution
- Continues execution even if individual files fail
Side Effects
- Modifies
LATEST_EVAL[]
global state - Clears and populates results file
- May output error messages for missing files
Error Handling
- Validates file paths before execution
- Reports missing files as errors with bug report guidance
- Individual file failures don't stop batch execution
TestPicker.save_test_results
— Methodsave_test_results(testset::Test.TestSetException, testinfo::TestInfo, pkg::PackageSpec) -> Nothing
Save test failures and errors from a test set to the package's results file.
Processes a test set exception containing failed and errored tests, formats them for display in the results viewer, and appends them to the package's results file. Each test result includes the test description, source location, detailed error information, and context.
Arguments
testset::Test.TestSetException
: Exception containing failed/errored teststestinfo::TestInfo
: Information about the test block that was executedpkg::PackageSpec
: Package specification for file organization
File Format
Each test result is stored as a line with components separated by separator()
:
- Test description (from
list_view
) - Cleaned source location (from
clean_source
) - Detailed error content (from
preview_content
) - Test context (from
context
)
Side Effects
- Creates results file if it doesn't exist
- Appends new results to existing file content
- Results are null-separated for fzf compatibility
Notes
Results are formatted specifically for consumption by visualize_test_results
and the fzf-based results viewer interface.
TestPicker.select_test_files
— Functionselect_test_files(query::AbstractString, pkg::PackageSpec=current_pkg()) -> Vector{String}
Interactively select test files using fzf based on a fuzzy search query.
Presents an fzf interface showing all test files for the package, with syntax-highlighted preview using bat. Users can select multiple files and the query pre-filters the results.
Arguments
query::AbstractString
: Initial fuzzy search pattern for filtering test filespkg::PackageSpec
: Package specification (defaults to current package)
Returns
Vector{String}
: Full paths to selected test files, empty if no selection made
Features
- Multi-selection enabled (can choose multiple test files)
- Syntax-highlighted preview of file contents
- Pre-filtered results based on initial query
- Returns full absolute paths ready for execution
Examples
# Select test files matching "math"
files = select_test_files("math")
# Select from specific package
pkg = PackageSpec(name="MyPackage")
files = select_test_files("integration", pkg)
TestPicker.separator
— MethodSeparator used by fzf
to distinguish the different data components.
TestPicker.test_mode_do_cmd
— Methodtest_mode_do_cmd(repl::AbstractREPL, input::String) -> Nothing
Execute test commands received in the test REPL mode.
Processes user input from the test mode, identifies the requested operation, and dispatches to the appropriate test execution or inspection function.
Arguments
repl::AbstractREPL
: The REPL instance for result visualizationinput::String
: The command string entered by the user
Commands Supported
- File queries: Run tests from matching files using fuzzy search
- Testset queries: Run specific test sets using file:testset syntax
- Latest evaluation: Re-run previously executed tests with "-"
- Result inspection: View test results with "?"
Side Effects
- May execute test code and modify package environments
- Updates the latest evaluation cache
- Can display test results in the REPL
- Outputs informational messages and errors
Error Handling
- Catches and reports
TestSetException
errors from test execution - Provides warning for non-interactive usage
- Handles unrecognized query formats
Examples
# In test mode:
# "math" # Run tests from files matching "math"
# "test_calc.jl:add" # Run "add" testset from test_calc.jl
# "-" # Re-run last tests
# "?" # View test results
TestPicker.testblock_list
— Methodtestblock_list(choices, info_to_syntax, display_to_info, pkg) -> Vector{EvalTest}
Convert user-selected test block choices into executable test objects.
Takes the selected display strings from fzf and converts them into EvalTest
objects that can be evaluated. Each test is wrapped in a try-catch block to handle test failures gracefully and save results.
Arguments
choices::Vector{<:AbstractString}
: Selected display strings from fzfinfo_to_syntax::Dict{TestBlockInfo,SyntaxBlock}
: Mapping from test info to syntax blocksdisplay_to_info::Dict{String,TestBlockInfo}
: Mapping from display strings to test infopkg::PackageSpec
: Package specification for test context
Returns
Vector{EvalTest}
: Collection of executable test objects ready for evaluation
TestPicker.visualize_test_results
— Functionvisualize_test_results(repl::AbstractREPL=Base.active_repl, pkg::PackageSpec=current_pkg()) -> Nothing
Interactive visualization of test failures and errors using fzf interface.
Creates a loop-based interface for browsing test failures and errors from the most recent test execution. Provides syntax-highlighted previews of stack traces and allows editing of test files directly from the interface.
Arguments
repl::AbstractREPL
: REPL instance for terminal operations (defaults to active REPL)pkg::PackageSpec
: Package specification for locating results (defaults to current package)
Features
- Two-level navigation:
- Browse failed/errored tests with stack trace preview
- Drill down into stack traces with source code preview
- Syntax highlighting: Uses bat for colored display of Julia code
- Editor integration:
Ctrl+e
opens files in configured editor - Source location: Shows relevant source code around error locations
Interface Controls
- Main view:
Enter
: Drill down into stack trace for selected testCtrl+e
: Edit the test fileEsc
: Exit the viewer
- Stack trace view:
Ctrl+e
: Edit source file at specific lineEsc
: Return to main test view
Prerequisites
- Test results file must exist (created by running tests with failures/errors)
- External dependencies:
fzf
,bat
, configured editor
Error Handling
- Warns if no results file exists
- Gracefully handles missing stack traces for simple failures
- Continues operation if source files cannot be located
Examples
# View results for current package
visualize_test_results()
# View results for specific package
pkg = PackageSpec(name="MyPackage")
visualize_test_results(Base.active_repl, pkg)