"When the time for Pentecost was fulfilled,
they were all in one place together.
And suddenly there came from the sky a noise like a strong driving wind, and it filled the entire house in which they were.
Then there appeared to them tongues as of fire,
which parted and came to rest on each one of them.
And they were all filled with the Holy Spirit
and began to speak in different tongues,
as the Spirit enabled them to proclaim." - Acts 2: 1-4

The Birth of the Church

Pope Benedict XVI taught on Pentecost 2009 what happened on the first Pentecost, when the Church was born:

"The Church throughout the world relives to today the Solemnity of Pentecost, the mystery of her own birth, of her own "baptism" in the Holy Spirit (cf. Acts 1:5), which took place in Jerusalem, 50 days after Easter, precisely on the Jewish feast of Pentecost. The risen Jesus told his disciples: "Remain in the city until you are given power from on high" (Luke 24:49). This happened in a perceptible way in the Cenacle, while they were gathered together with Mary, the Virgin Mother, in prayer. As we read in the Acts of the Apostles, suddenly that place was invaded by a strong driving wind, and tongues like fire came to rest on the heads of all those present. The Apostles went out then and began to proclaim in different languages that Jesus is the Christ, the Son of God, dead and risen (cf. Acts 2:1-4). The Holy Spirit, who with the Father and the Son created the universe, guided the history of the people of Israel and spoke through the prophets, who in the fullness of time cooperated in our redemption, who at Pentecost descended upon the nascent Church and made it missionary, sending it to proclaim to all peoples the victory of divine love over sin and death."  Source: ZENIT News

The Bible records the story of Pentecost in Acts chapter 2.

Mary's Role on Pentecost

The Bible singles out the Virgin Mary as having a special role on Pentecost because it specifically names her and the Apostles, the leaders of the Church, in Acts 1:13-14 as being united in prayer for the Holy Spirit.

Pope Benedixt XVI taught us to be like the early Church, open to receive the benefit of her prayerys which she lifts up to God in Heaven. He taught the significance of this on Pentecost Sunday, May 27, 2007 in Rome, saying, "Dear brothers and sisters, the first Pentecost took place when Mary Most Holy was present amid the disciples in the Upper Room in Jerusalem and prayed. Today, too, let us entrust ourselves to her maternal intercession, so that the Holy Spirit may descend in abundance upon the Church in our day, fill the hearts of all the faithful and enkindle in them the fire of his love."

Pentecost Continues

"And the Church continues to live on that same strength which enabled it to begin and to grow. Pentecost is the original event but also a permanent dynamism" - Pope Benedict XVI, Homily, October 10, 2010

In the Footsteps of the Early Church: The Pentecost Novena

Following the example of the early Church after Jesus ascended to Heaven, many Christians pray the Novena to the Holy Spirit during nine days between Ascension Thursday and Pentecost Sunday.

The International Catholic Charismatic Renewal Services office at the Vatican encourages Christians to unite in prayer through Pentecost for the Nations.  This call for prayer encourages people to pray for a "spirituality of Pentecost" to flourish worldwide.

The Holy Spirit coming amidst the angels, image from the church of Trinita dei Monti in Rome