NPL offers powerful tools for unit testing, yet it presents challenges when attempting to run more sophisticated tests that encompass complex user scenarios beyond individual or grouped protocols.
A common hurdle arises when protocols are created dynamically within the application’s workflow. Currently, the testing capabilities in NPL do not support capturing which protocols are generated during a specific test scenario unless they are explicitly referenced by an object. This limitation forces us to introduce global shared mutable state across the application to capture these protocols so they can be located later. In real-world applications, these protocols are exposed through the NPL API, which provides the capability to query protocols by type and other fields as needed.
Having a tool to similarly expose protocols created during test execution would be incredibly valuable. Ideally, this tool would enable querying protocols by type, ID, or other parameters, simplifying the testing process for complex scenarios. Even a straightforward list of all protocols by type would greatly benefit developers working on more extensive, scenario-based tests.
If anyone knows of such functionality within NPL or can suggest a workaround, I’d appreciate the insight!
Hi Sergey, could you provide a small motivating example please?
The reason I’m asking for this is because it sounds like you might be coupling the tests too much to the implementation. So it would be useful to see a small example scenario.
Stuart, thanks for your reply. Here’s a motivating example to illustrate the limitations we’re facing in testing more complex scenarios with NPL.
In this example, we have a simple structure for placing and fulfilling trades via orders. The challenge here is that while trades are accessed through the orders list, in more advanced scenarios, we can’t rely on the test framework to retrieve them directly, especially if that list is absent or orders are nested further within workflows. This scenario represents a common hurdle when protocols are created dynamically or when scenarios evolve beyond unit testing, leading to an uncomfortable coupling where tests are overly dependent on returning all outputs. Here’s the sample:
package npl;
protocol [TradingSystem]Trade() {
@api
permission[TradingSystem] execute() {
}
}
@api
protocol [OrderManager]Order(var amount: Number) {
var trades : List<Trade> = listOf<Trade>();
init {
info("Order placed");
}
@api
permission[OrderManager] fulfill() {
// make some trades
trades = trades.with(Trade['TradingSystem']());
}
}
@api
protocol [Trader]OrderSystem() {
var orders : List<Order> = listOf<Order>();
@api
permission[Trader] addOrder(order: Order) {
orders = orders.with(order);
}
}
protocol [Trader, OrderManager]System() {
var orderSystem: OrderSystem = OrderSystem[Trader]();
permission[Trader] placeOrder(amount: Number) {
orderSystem.addOrder[Trader](Order[OrderManager](amount));
};
}
@test
function testMyCode(test: Test) -> {
var system = System['Trader', 'OrderManager']();
system.placeOrderorderSystem.orders.get(0).fulfill['OrderManager']();
system.orderSystem.orders.get(0).trades.forEach(function(trade: Trade) -> trade.execute['TradingSystem']());
}
In this code, we see that Trade is a dynamically generated protocol. Accessing these trades is straightforward here since they’re linked to orders, but in real-world scenarios, they might be harder to access directly. This example also demonstrates how the test is forced to capture outputs manually, which can become cumbersome in more complex flows.
The current setup often forces developers to introduce additional global shared mutable state to capture dynamically created protocols so they can be accessed later for testing. The ability to query protocols by type or ID within the testing framework could streamline tests for these scenarios, making it easier to verify complex workflows without adding unnecessary coupling or global state.
Thanks @Sergey_Kisel . I think I get what you’re saying - it’s hard to pull out a protocol that has been created inside another protocol in order to perform some action on it, in this case perform execute on a specific Trade protocol.
So, I think what you’re asking for is code introspection or reflection only for tests?
I think this can be useful, but as I said, we would need to be wary of writing tests that are too coupled to the implementation.
If you want to do this now (find an arbitrary protocol by some attribute) then I believe the only option is to interrogate the NPL from the outside using an integration test.