When they were children, both Peter and Olivia were patients during the Cortexiphan trials run by Walter Bishop and William Bell. Olivia ran away to a field of white tulips, where Peter found her and encouraged her to tell Dr. Bishop about her abusive stepfather. (Subject 13)
Olivia and Peter first meet in Iraq when Olvia needs him to take her to his Father, Walter Bishop. Altough he doesn't want to help her at first, he changes his mind after she treatens him with an FBI file wich is later reaveald to be fake. When their first case is closed Peter wants to leave the country but stays out of pity for Olivia.
In the fourth episode Peter clearly states that he wants to leave the country, but after being kidnapped he changes his mind and tells Olivia that he will stay until he finds answers to where all the strange things that are happening originate from.
Throughout the first season, Olivia and Peter develops from a friendship to Peter clearly having feelings for her, but she never seems to reciprocate because she is getting over her dead fiancee, John Scott.
In Season 2 finale(When They were in the alternate universe).Olivia Said to Peter; "You must come back Because you belong with me".
In Season 4 finale,It was reavealed that Olivia is pregnant with Peter's Child
"Peter,I love you" -Olivia
"I love you too" -Peter
"Peter,I'm pregnant" -Olivia