I Confused Things Between Us

I am a 23 year old girl. There is a boy in my company with whom we liked each other more than friends over time. For a few months we just wrote to each other, and when we found ourselves in the company, we almost didn't talk to each other, because I was ashamed, and maybe it was the same with him. Otherwise, he had asked me several times in plain text if I liked him too and if I wanted to go out on a date, and each time I said yes, because that was the truth. So, last week he invited me to go out alone, but I refused. I really didn't have the opportunity then, because he asked me almost at the last minute and I already had other plans. A few days later he invited me again and I did not go again, but this time simply because I was scared. I felt very unsure how the meeting would go, whether it would be awkward, and I wondered how things would turn out after we met. I've had boyfriends before, but so far I haven't worried so much. I didn't know, but that night he was leaving for work for two weeks. The next day I wrote to him to apologize, I told him that I really didn't want it to happen and that when he came back we would really see each other. He replied that there was no problem, and I was behaving as usual, but I still felt that things were not quite right. Because only I look for him, he answers, sends me kisses, compliments me and then disappears, stops writing to me. I shared it with a friend and she said she didn't know what she wanted. But I think it's my fault, because every time he asked me, I said I wanted to see him, and in the end I refused twice. He even invited me the second time to a class where he knew he was comfortable with me and I had no other plans ... I feel terrible that I am such a coward. I don't know what to do. Should I give him time? Should I look for him again? And most of all, should I invite him to see me this time?