We have a functional test framework written in powershell that we use to verify the NuGet's behavior in Visual Studio.
After setting up your environment and running the experimental instance of VS, you can run the tests by running the cmdlet Run-Test. You can run a specific test by specifying the name of the test as a parameter to Run-Test (we also show IntelliSense for test names).
All tests are located in test\EndtoEnd\tests. Tests are grouped by functionality (it's purely convention based) so all tests that test NuGet's install functionality resides in the install.ps1 file. Tests are regular powershell functions that start with the name "test":
function Test-Foo {
"Hello World"
}
After writing the above test in any file under the test folder, you can run it in the console by typing Run-Test Foo (notice how we exclude the Test- prefix from the name of the test while running it). The test runner loads all scripts under the tests folder, imports all the functions with the Test- prefix into the environment for execution.
Since this framework is mostly for testing NuGet within visual studio, each test is passed a context object that has 2 paths:
To avoid checking in binaries into the source tree, the functional test framework allows two ways of generating packages on the fly. Under Tools\NuGet\GenerateTestPackages there is a tool that takes one of the two forms of inputs and creates binaries from it. Specific tests can specify these input files in their RepositoryPath
1) DGML DGML is a dsl for generated directed graphs. In our case, this specifies the dependency chain of packages and builds a set of packages from it with default set of package content. This comes in handy when trying to test complex package graph behavior.
2) NuSpec files If you need more fine grained control over what each package's content and metadata is, you can check in multiple nuspec files of the format Id.Version.nuspec. If the package specifies a files node, the generator tool would look for it under RepositoryPath\files*full path to file* path. If no file exists at this location, the generator would create an appropriate file and include that in the resulting package.
Tests aren't useful unless you're actually testing interesting behavior. Just like anything else in the console, tests have full access to $dte, but we also have some useful helper functions that aid you when trying to interact with VS.
Assert-AreEqual $Expected $Actual
Assert-True $Value
Assert-False $Value
Assert-Fail "Failed :("
# Assert that the exception message matches the expected exception message.
Assert-Throws { throw "foo" } "foo"
Assert-PathExists "c:\foo"
$project = Get-Project MvcApplication1
Assert-Reference $project System.Web
# Assert that there are no errors.
$project = Get-Project SilverlightApplication
Assert-Build $project
$project = Get-Project MyProject
# Assert that this project has any version of package A.
Assert-Package $project A
# Assert that this project has package A 2.0.
Assert-Package $project A 2.0
# Assert that the solution has package A.
Assert-SolutionPackage A
# Assert that the solution has package A 1.0.
Assert-SolutionPackage A 1.0
When building tests it's useful to create new projects in each test for better isolation (each test is run in a new solution). We have helpers for project creation and helpers that make common DTE operations easier.
We store project templates locally so we don't have to depend on the ones installed in VS. Project templates are stored in test\EndToEnd\ProjectTemplates.
# The project name is the name of a folder under the project templates folder (without the .zip)
New-Project ClassLibrary
There are also a number of shortcuts so that we don't have to use New-Project to create projects:
$project = New-ClassLibrary
The above will create a new class library and return the project instance.
You might have realized that New-Project doesn't take any parameters that have to do with where the project is physically created on disk, this is because we create a solutions and projects in test\EndToEnd\bin. This makes it easy to create new projects without having to think about where things need to go. The other benefit of putting then in the bin folder is that it means they are ignored in source control. You can however pass a name if you like.
Each project and solution end up with a random name, e.g. Solution_06c6.
You can also create Solution folders and create projects within solution folders:
# Create a solution folder called foo
$solutionFolder = New-SolutionFolder foo
# Create a project within that folder
$project = $solutionFolder | New-ClassLibrary
# Build debug configuration
Build-Project $project Debug
# Build release configuration
Build-Project $project Release
# Removes item from the project and disk
Remove-ProjectItem $project Default.aspx
# Gets the project item for jquery
$projectItem = Get-ProjectItem $project Scripts/jquery.js
# Get the physical path for web.config in the views folder
$fullPath = Get-ProjectItemPath $project Views/web.config
# Get the output path of the project (based on current configuration)
$outputPath = Get-OutputPath $project
# Get the path to the project root
$projectPath = Get-ProjectDir $project
# Get-PropertyValue $project AssemblyName
# Get the assembly reference for System.Web
$assemblyReference = Get-AssemblyReference $project System.Web
Returns a list of errors in the current error list.