To delete a connection select a connection and choose Delete Connection from context menu or press Delete.

Note: A connection must be closed in order to delete it.