Quest are complete, but one task in journal "Talk to Pyrite again" are still marked with green "active" diamond instead purple one like completed tasks have. Exclamation mark over Pyrite's head or Heartland's portals still hanging and interfere other tasks. Talking to Pyrite not helps.
This issue has been addressed in other tasks. The fix for the problem was made after some players had finished the quest, so the objective marker is stuck. Currently the only fix for any issue like this would be to delete and remake your pony.
Yes, but that's your free choice to do it or not. The game is still in an early state. I advice you to show patience. At least the issue you have doesn't seem to be permanent, but it may take months.