Unfortunately, when you close apps in Windows 8.1, they aren't actually closed.
As you know, dragging an app towards the bottom of the screen in Windows 8 will close the app. However, in Windows 8.1 the app will just be in a suspended state when you perform the app close gesture.
Maybe you discovered this by seeing your “closed” app in the task manager when you were poking around in the task manager as I had previously. To fully close your app:
- Move your mouse or finger (if you have a touch device) to the top of the app
- Click or tap the top of the app and hold while dragging the app towards the bottom of the screen
- Hold the app towards the bottom and do not release your hold
- After the app spins around, release your hold to close
If you're feeling nostalgic, Windows 8.1 Update 1 adds a X in the top right to close your apps. Also, the standard keyboard shortcut ALT+F4 still works as well.