smart.router is sold and supported by your internet provider. They installed it, they can see your router, and they are the fastest way to get something fixed.
That is the app doing its job. Your traffic has already moved to another connection and your internet is working. The line that failed is your provider's to fix, and they have been told.
On the WAN tab, drag a connection to the top. The router applies it within about ten seconds and you get a confirmation. The old connection stays ready to take over.
The test measures the connection that is carrying traffic at that moment. If you are running on a backup, you are measuring the backup. Check which one shows as active first.
Device inventory is a separate package. Ask your provider to add it.
The Nate tab answers questions about your own network from live data, and can run bounded checks for you. It only ever reads, and only ever about the router you are looking at.
smart.router for iPhone shows the connections on your router, live and historical traffic, the devices on your network, a speed test, a VPN back into your own network, and the assistant. It needs a router already enrolled by your provider; there is nothing to configure yourself.