To edit/remove Tags or Segments, first navigate to Admin Settings (Available to administrators only.)
Select Tags from the left column.
Edit or remove from the right:
The same options can be accessed from Segments.
Edit or delete existing segments from the right:
See this in action in the video below.