dev\microsoft-agents-testing improvements

[Rodrigo Brandão (rodrigobr-msft)]

This pull request primarily updates the integration test and documentation code to standardize scenario and agent initialization patterns, and to improve testability and maintainability. The changes include switching from direct class instantiation to using the AiohttpScenario.create and ActivityHandlerScenario.create factory methods, refactoring agent initialization functions to be synchronous, and enhancing the fidelity of test doubles for authentication. Additionally, minor documentation and test improvements are included.

Test and Scenario Initialization Refactor:

• Replaced direct instantiations of AiohttpScenario and ActivityHandlerScenario with their .create factory methods throughout test and sample files, ensuring consistent scenario setup and improved extensibility. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
• Updated all agent initialization functions (e.g., init_agent, init_echo, etc.) in both tests and documentation samples to be synchronous (def instead of async def), matching the expected signature for scenario creation. [1] [2] [3] [4] [5] [6] [7] [8] [9]

Authentication and Test Double Improvements:

• Enhanced the _FakeUserTokenClient test double to properly subclass UserTokenClientBase and implement all required async methods, improving the fidelity of authentication flow tests. [1] [2] [3]
• Changed init_agent in some authentication and streaming tests from async def to def for consistency and compatibility with the new scenario creation pattern. [1] [2] [3]

Test and Documentation Consistency:

• Updated references to application configuration in JWT validation tests to use dictionary-style access (app["agent_configuration"]) instead of attribute access, ensuring compatibility with aiohttp best practices. [1] [2]
• Minor documentation corrections, such as removing references to deprecated methods in the API markdown file.

Booking and Dialog Scenario Updates:

• Refactored dialog test scenarios to use the new ActivityHandlerEnvironment and updated handler factories accordingly, improving clarity and reducing boilerplate. [1] [2] [3]

These changes collectively improve the maintainability, clarity, and extensibility of the test and documentation codebase.

[Copilot]

Pull request overview

This PR refactors the dev/microsoft-agents-testing scenarios and related tests/samples to standardize scenario construction, simplify agent setup callbacks, and improve fidelity of authentication test doubles. It also updates docs and integration tests to align with the new scenario patterns and removes deprecated streaming helpers from the testing client APIs.

Changes:

• Standardize scenario construction by switching call sites to AiohttpScenario.create(...) / ActivityHandlerScenario.create(...) and making setup callbacks synchronous.
• Improve in-process scenario infrastructure (lazy environment materialization, optional anonymous connections, and ActivityHandlerScenario parity with AiohttpScenario).
• Update auth/integration tests and docs to match the new APIs (including removing send_stream / ex_send_stream references).

Reviewed changes

Copilot reviewed 37 out of 40 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/_common/testing_objects/mocks/mock_user_token_client.py	Minor formatting cleanup for AsyncMock setup.
libraries/microsoft-agents-hosting-core/microsoft_agents/hosting/core/channel_service_adapter.py	Add cast import for typed access to invoke response stored in turn state.
libraries/microsoft-agents-hosting-core/microsoft_agents/hosting/core/_http_adapter_base.py	Default missing claims identity to an explicit Anonymous ClaimsIdentity.
dev/microsoft-agents-testing/tests/test_scenario_registry_plugin.py	Use AiohttpScenario.create and sync setup; configure omit-connections for local tests.
dev/microsoft-agents-testing/tests/test_pytest_plugin.py	Use AiohttpScenario.create and sync setup in plugin tests (echo/counter scenarios).
dev/microsoft-agents-testing/tests/test_aiohttp_scenario.py	Update unit tests to validate new create/setup semantics and lazy env materialization.
dev/microsoft-agents-testing/tests/test_aiohttp_scenario_integration.py	Update integration tests to use AiohttpScenario.create and omit connections where appropriate.
dev/microsoft-agents-testing/tests/test_activity_handler_scenario.py	Add tests for new/updated ActivityHandlerScenario + environment behaviors.
dev/microsoft-agents-testing/tests/manual.py	Update manual runner example to AiohttpScenario.create with sync setup.
dev/microsoft-agents-testing/tests/core/test_agent_client.py	Remove tests for send_stream/ex_send_stream that are no longer supported.
dev/microsoft-agents-testing/tests/cli/test_cli_integration.py	Keep CLI integration examples in comments aligned with the new create API.
dev/microsoft-agents-testing/microsoft_agents/testing/utils/helpers.py	Add expect(...) helper wrapping ExpectBase for single-item or sequence input.
dev/microsoft-agents-testing/microsoft_agents/testing/utils/init.py	Export new expect helper from utils.
dev/microsoft-agents-testing/microsoft_agents/testing/source_scenario.py	Expand SourceScenario docstring with a usage example.
dev/microsoft-agents-testing/microsoft_agents/testing/mocks/mock_cloud_adapter.py	Add new mock helper module (currently incomplete/syntax-invalid per review comments).
dev/microsoft-agents-testing/microsoft_agents/testing/mocks/anonymous_connection_manager.py	Add AnonymousConnectionManager helper for tests that need anonymous tokens.
dev/microsoft-agents-testing/microsoft_agents/testing/mocks/init.py	Package marker for mocks module.
dev/microsoft-agents-testing/microsoft_agents/testing/core/fluent/expect.py	Allow ExpectBase to accept a single dict/BaseModel (auto-wrap) or a sequence.
dev/microsoft-agents-testing/microsoft_agents/testing/core/agent_client.py	Remove send_stream/ex_send_stream APIs from AgentClient.
dev/microsoft-agents-testing/microsoft_agents/testing/cli/scenarios/basic_scenario.py	Convert CLI-registered scenarios to AiohttpScenario.create with sync init.
dev/microsoft-agents-testing/microsoft_agents/testing/cli/scenarios/auth_scenario.py	Convert auth CLI scenario to AiohttpScenario.create with sync init.
dev/microsoft-agents-testing/microsoft_agents/testing/aiohttp_scenario.py	Refactor AiohttpScenario to support create/from_app, lazy env creation, and omit-connections mode.
dev/microsoft-agents-testing/microsoft_agents/testing/activity_handler_scenario.py	Refactor ActivityHandlerScenario to align with AiohttpScenario patterns and JWT middleware behavior.
dev/microsoft-agents-testing/docs/samples/utilities.py	Update docs sample to use AiohttpScenario.create with sync init.
dev/microsoft-agents-testing/docs/samples/transcript_formatting.py	Update transcript formatting sample to use AiohttpScenario.create with sync init.
dev/microsoft-agents-testing/docs/samples/scenario_registry_demo.py	Update registry demo to use AiohttpScenario.create scenarios.
dev/microsoft-agents-testing/docs/samples/quickstart.py	Update quickstart doc sample to use AiohttpScenario.create.
dev/microsoft-agents-testing/docs/samples/pytest_plugin_usage.py	Update pytest plugin usage sample to use AiohttpScenario.create.
dev/microsoft-agents-testing/docs/samples/multi_client.py	Update multi-client sample to use AiohttpScenario.create.
dev/microsoft-agents-testing/docs/samples/interactive.py	Update interactive sample to use AiohttpScenario.create.
dev/microsoft-agents-testing/docs/API.md	Remove docs entries for removed send_stream/ex_send_stream.
dev/integration/uv.lock	Update integration lockfile dependencies (including fastapi/starlette/anyio additions).
dev/integration/tests/test_streaming_response.py	Update streaming response integration test to AiohttpScenario.create.
dev/integration/tests/scenarios/quickstart.py	Update quickstart integration scenario init to be synchronous.
dev/integration/tests/scenarios/init.py	Switch scenario loader to AiohttpScenario.create.
dev/integration/tests/jwt_validation/test_aiohttp_jwt_validation.py	Use aiohttp app dict-style access for agent_configuration.
dev/integration/tests/auth/test_oauth_continuation.py	Improve fake user token client fidelity by subclassing UserTokenClientBase and implementing required async methods.
dev/integration/tests/activity_handler/dialogs/test_booking_dialog.py	Update dialog scenario creation to ActivityHandlerScenario.create using ActivityHandlerEnvironment.
dev/integration/tests/activity_handler/dialogs/scenario.py	Update dialog scenario factory to ActivityHandlerScenario.create using ActivityHandlerEnvironment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 38 out of 40 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (5)

dev/microsoft-agents-testing/microsoft_agents/testing/aiohttp_scenario.py:184

• MemoryStorage() is instantiated twice in _default_env_factory; the first instance is immediately discarded. This is confusing and slightly wasteful; keep a single storage initialization.

        connection_manager: Connections
        if omit_connections:

libraries/microsoft-agents-hosting-core/microsoft_agents/hosting/core/_http_adapter_base.py:114

• This type annotation formatting is not Black-stable (Black will rewrite it because parentheses in annotations are not preserved in the AST). Since CI checks Black formatting under libraries/, this is likely to fail formatting checks.
  dev/microsoft-agents-testing/microsoft_agents/testing/core/fluent/expect.py:60
• This line is a blank line containing trailing whitespace. It’s better to remove it to avoid noisy diffs and whitespace lint warnings.
  dev/microsoft-agents-testing/microsoft_agents/testing/utils/helpers.py:13
• PEP 8 expects two blank lines before a top-level function definition. Adding an extra blank line here avoids common flake8 E302 warnings and keeps the style consistent with most modules.

ModelT = TypeVar("ModelT", bound=dict | BaseModel)

def expect(items: ModelT | Sequence[ModelT]) -> ExpectBase[ModelT]:

dev/microsoft-agents-testing/microsoft_agents/testing/aiohttp_scenario.py:57

• logger is defined but never used in this module. Removing it will also make import logging unused, so that import should be dropped as well (if there are no other logging uses added).

logger = logging.getLogger(__name__)

[Copilot]

Pull request overview

Copilot reviewed 38 out of 40 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

libraries/microsoft-agents-hosting-core/microsoft_agents/hosting/core/_http_adapter_base.py:114

• The claims_identity annotation is formatted in a way that Black will rewrite (and Black checks run for libraries/). This will likely cause formatting drift/CI failures; prefer the standard single annotation form.