Skip to Content
We've launched the Church Streaming License, a modern license that provides a complete solution for today's churches.Learn More

Blog

NEW RELEASES | Week of 9/26/22

Cody Carnes | God Is Good!

God is Good! was captured this summer during a special two-night worship event at The Belonging Co in Nashville. The record features singles "Good (Can't Be Anything Else)," "Ain't Nobody," "Forever & Amen," and special guests Kari JobeNatalie Grant, and Benjamin William Hastings

JJ Hairston | Believe Again

Hairston shares: “Previously, my creative process evolved from the worship perspective. Although I am still a worshiper, as a pastor, I teach more. Now, each musical project starts with a thought or a scripture. I spend a great deal of time thinking about what I want people to get from each song. Faith is a choice, and we hope the album will inspire people to choose God."

Josh Baldwin | Where The Glory Is

'"Where The Glory Is" from Josh Baldwin reminds us that His glory is all around us and we have hope that in every area of our lives, we will never walk alone. This song is the title track of his upcoming album out on October 21st! 

Mosiac MSC | Comfort

Mosaic MSC is the worship collective of Mosaic Church in the heart of Hollywood. Their latest single "Comfort" is a reminder of the relief we find in Jesus's presence that is both fierce and tender. Resources available now! 

+ MORE NEW RELEASES:

Community Music | God of Resurrection

Evan Craft | Someone Like Me

North Coast Church | Prison Escape

SEU Worship | Heart Cry

 

MORE FROM MULTITRACKS.COM

APPLE MUSIC + SPOTIFY PLAYLISTS:

 

New Podcast Episode with Yancy

Yancy is a Dove Award-winning worship leader, songwriter and producer of hundreds of songs that make Jesus loud. She is a powerhouse at leading and teaching kids how to worship and creates popular worship resources for churches and homes around the world.

Kristian and Yancy caught up at Experience Conference on the authenticity of kids worship, how valuing kids ministries permeates into all aspects of spiritual life, the importance of accurate reporting and her latest book, Sweet Sound: The Power of Discipling Kids in Worship. We champion Yancy’s heart for kids ministry and hope this conversation encourages intentionality and prioritization for this area of the Church!

Apple Podcasts

Spotify

NEW RELEASES | Week of 9/19/22

Bethel Music | Simple

Bethel Music shares the heart behind their album saying, "In this season for Bethel Music, we keep coming back to the simple message of the Gospel. This album focuses on the simplicity of intimate worship with God. We want to give the Lord our most honest, vulnerable selves and this album guides us into that desired heart posture before the Lord."

UPPEROOM | Moments: Image 005

"Image 005 is a compilation of special times of live worship we have had in our house of prayer and at our weekend church services at UPPERROOM. It is the fifth Moments project we have released and includes original songs our team have written, cover songs and spontaneous moments of prophetic worship," shares UPPERROOM. "These special moments are becoming more frequent than rare, and we are excited to release an EP of moments every month through the rest of the year. We believe these corporate songs and moments of intimate encounter with the Lord will be a blessing to you."

Alexander Pappas | American Dream

Pappas shares: "I pray songs like 'Older' and 'High Hopes' would inspire you to grow, that 'Keep Your Word' and 'Goodbyes' would fill your faith, that 'Does Anyone Else Feel Like Me?' would lead you towards a newfound vulnerability, trusting and knowing that you're not alone and finally, that 'Million Ways' and 'American Dream' would point you towards the magnitude of what a good God He is."

 

+ MORE NEW RELEASES:

Ben Fuller | Who I Am

Blanca | The Heartbreak and the Healing

Gateway Worship | I Need You - MultiTracks.com Session

Matt Redman | Son of Suffering

 

MORE FROM MULTITRACKS.COM

APPLE MUSIC + SPOTIFY PLAYLISTS:

 

A few words about Sunday's downtime.

If you use MultiTracks.com, ChartBuilder, or Playback you might have noticed our API had some critical performance issues this past Sunday beginning around 7:30 am CST We understand that you trust our products to always work, especially on a Sunday morning, without disruption. This is our standard, and we apologize that we did not meet this standard last Sunday. The stability of each of our products is and has been our top priority and we know that we let many of you down this weekend. Please be assured that our team takes this very seriously, and we are passionately committed to being a platform and a partner you can trust.  

To that end, we want to disclose what went wrong and provide confidence that we are addressing the issue. Simply put, we had a bug in our system that caused a cascade of issues that were not all related, but the result is that they caused a significant load on our servers that resulted in downtime on Sunday morning. We have identified the issue and put measures in place to ensure that this issue does not happen in the future. 

As a result of what happened, here are the changes we are implementing to prevent them from happening in the future: 

  • We have already begun significant infrastructure investments to scale accordingly should an issue like this happen in the future. On Sunday there were several problems which on their own could have been addressed without affecting customers but together they cascaded into a severe degradation in our service. We are adding additional servers and making improvements to our monitoring as a result. 

  • We are implementing further automated testing for elevated load scenarios. This includes mirroring closer to production datasets and peak traffic scenarios. 

  • We are adding better offline support to our apps to handle transient network errors. Many of you reported being logged out unexpectedly in ChartBuilder yesterday. That should not be the case and this will be addressed. You should be able to freely use both Playback and ChartBuilder without any loss of connection to the network for content you had previously downloaded. 

If you are interested in the technical reasons why we had issues, please read on. 

 

What exactly went wrong? 

We utilize SQL Server to service requests for many parts of our applications from authentication details to downloading MultiTracks metadata into Playback. Last week we deployed an update to a database function that updates content in your library. At MultiTracks we have several layers of review before code gets deployed into production. We employ an internal developer code review that checks for performance, security, and functional acceptance. From there, we employ a dedicated QA team that tests each issue for acceptance as well. The code mentioned passed both of those without issue. However, we did not realize that under strain of significant server load that the code could cause cascading SQL performance issues. 

 

The Incident 

Due to the spike in activity from Sunday morning, at 7:30 AM our SQL Server reached peak capacity. This caused a significant increase in the load to our servers. For example, this meant requests that typically complete in milliseconds were now taking 10s of seconds or longer to complete. This increased load caused some requests to start failing. As a result, our apps began sending additional traffic to satisfy the data they needed to complete actions customers were taking in their apps. Our apps employ an automated retry for network requests to mitigate transient network errors typically seen on network requests. In this scenario, a single user action could, in some cases, result in several retries occurring behind the scenes.  

As requests rapidly increased in quantity, they cascaded among themselves, and the result is that traffic grew exponentially beyond what our servers could handle. At that point, our web cluster exhausted the number of ports it could connect with, and we stopped sending requests to our caching tier. Once the caching tier was exhausted, we fell back to SQL Server again which exacerbated an already demanding situation.

Though our team began working diligently to solve the issue, in the confusion of diagnosing multiple issues simultaneously, it took us until 11:30 AM CST before everything had been resolved across our platform. 

 

Key Takeaways 

  • We are dramatically increasing the scale of our SQL Server infrastructure to cope with significantly more load than is typical of our infrastructure today. This will provide a significant buffer if this issue were to ever occur again, though we are also taking additional steps to prevent that possibility. 

  • We are adding additional capacity and scale to our webfarm. 

  • We discovered a bug with ChartBuilder that caused customers to be logged out inadvertently, which should not be the case and will be addressed.  

  • We are developing additional resiliency with our apps to cope with unexpected network responses or network performance degradation. 

  • We are adding additional alerts to ensure the highest performance standards are met. 

  • We are adding additional testing that will ensure performance at extremely high peak load times. 

 

Conclusion

It is important to us that we keep our products running smoothly, and that you know that you can trust them to work for you every Sunday. We feel confident that these steps will mitigate the risk of any future incident of this nature.  

Phillip Edwards 
MultiTracks.com | Founder 

NEW RELEASES | Week of 9/12/22

Hillsong Worship | Team Night

"Team Night is not about us - it's a chance to gather as a worship & creative team, to prepare for our church's worship services. In order to facilitate these services well, we need a strong, healthy, functional team. No matter what a Team Night might look like, the outcomes for us are always the same: gather together as a creative team, rehearse for Sunday services, train our team and grow their craft, build a culture of excellence and creativity, build community, inspire our team to grow in their relationship with Christ."

Cody Carnes x Kari Jobe | Forever & Amen

Cody shares: "Forever & Amen" is a song about prayer and has reminded me of the power of the secret place. One of my favorite scriptures has become Psalm 116:2, 'Because He bends to listen, I will pray as long as I have breath!' What a powerful picture. God bending to listen to our prayers. We created this song with new melodies around the Lord’s prayer and with extra space, to help create an atmosphere for that time of prayer with God. I pray it reminds you how important it is to do what Jesus teaches us in Matthew 6: 'When you pray, go into your most private room, close the door and pray to your Father who is in secret, and your Father who sees in secret will reward you.'”

Gateway Worship | Here In Your House

When the presence of God fills His house, our praise and joy cannot be contained! “Here In Your House” is out now and we invite you to join us in this song of freedom. Let’s invite God to do His work in us and through us. Surely the Lord is in this place and we’ll worship Him in His house!

Passion | Live From Camp

Featuring CrowderSean Curran, and more, Passion Camp is all about middle and high school students gathered together for four days of worship, teaching and community. The album captures Passion's desire for wanting middle schoolers and high schoolers to leverage their lives for what matters most—the fame and glory of Jesus!

Tasha Cobbs Leonard | Jesus, Lover of My Soul (feat. The Walls Group)

Featuring The Walls Group, this song gives praise to God's promises and healing. He is our strength when we are broken, and His love is never ending. Glorifying His name, "Jesus, Lover of My Soul," is an anthem of how God breaks chains and sets us free! 

 

+ MORE NEW RELEASES:

Hope Darst | If The Lord Builds The House (feat. Jon Reddick)

KXC | I Will Not Stay Silent

Saddleback Worship | Not Gonna Wait

We Are Messengers | Come What May

 

MORE FROM MULTITRACKS.COM

APPLE MUSIC + SPOTIFY PLAYLISTS:

 

NEW RELEASES | Week of 9/5/22

Bethel Music | Tend

In John 15 Jesus says, “I am the true vine, and my Father is the gardener.” How the Lord cares for us is shown in how He tends the soil of our hearts. And our willingness to receive His pruning shows the depth of our relationship with Him. "Tend" is a song of surrender. That as we surrender that which comes in between, we grow in closeness with our loving Father.

Chris Tomlin | Always

Chris Tomlin has released his new album, Always! "I am so excited to see how He uses these new songs of worship. My prayer for these songs is that people are reminded Jesus is the same yesterday, now, and always.”

Israel & New Breed | Wrong About You (feat. Joe L. Barnes)

The third song on Israel & New Breed's upcoming album, "Wrong About You," features driving acoustic guitars, heart-thumping rhythm sections and a breathtaking choir. Resources are available now!

Tribl x SONS THE BAND | Live From 1971

Partnering with Tribl on their debut album, Live From 1971, they are able to capture that desire to create music that has its own identity. They share: “We wanted to push the limits and create something that was truly different but is still simple enough to worship to in any season.”

Leeland& TAYA | Heart & Flesh

Leeland and TAYA collaborated to create their new song "Heart & Flesh." Inspired by Psalm 84, the song exists to remind the people of God that He is in the middle of the chaos and upheaval in our lives. "Heart & Flesh" resonates in the tension, that though we cannot always change our circumstances, we can focus on the One who can.

+ MORE NEW RELEASES:

Brad & Rebekah | Life Again

Centricity Worship | God Is Love

Gas Street Music | Up To Something

Erik Nieder | I Will Worship

 

MORE FROM MULTITRACKS.COM

APPLE MUSIC + SPOTIFY PLAYLISTS:

 

New Podcast Episode with Luke Anderson

Our closing guest for SUMMER OF SOUNDS is none other than Luke Anderson. Luke was a part of Elevation Church for over 10 years and has played drums on several Elevation Worship albums and tours. Luke is also a drum instructor, a sessions musician, and creates drum samples for MultiTracks.com.

Mike and Luke chat about Extremely Small Drums (our April Fool’s collab) & not taking yourself too seriously, his tenure at Elevation Church, how the audition process is still necessary and refining in the worship space, and urges against letting the thing your called to become your identity. This is a special one! Thanks so much to Mike Graff of Coresound for hosting this series. We’ll be back for Season 4 this fall!

Apple Podcasts

Spotify

NEW RELEASES | Week of 8/29/22

Israel & New Breed | NewBreed Anthem + Miracles Signs & Wonders

Two singles from Israel & New Breed's upcoming album, Worship Anywhere, are available now! Both "NewBreed Anthem" and "Miracles Signs & Wonders" offer a joyful, stomp-clap singalong that sets the tone for the album, as well as a dynamic, feel-good groove.

Planetshakers | Greater

“When you read the Psalms about praising God, it’s very descriptive,” notes Joth Hunt. “Clap your hands all you people, shout to God your voice of triumph, rejoice before Him, dance before the Lord with all your might – a lot of these scriptures paint a picture of what praise should be like. And it’s obviously a celebration. My encouragement for the Church globally is to not lose that celebration of praise,” continues Hunt. “Whenever we do it, it really does break through the atmosphere...almost breaks people open, dusts them off and gets them into a place where they can receive from God."

Phil Wickham | Behold (feat. Anna Wilson)

"What a joy it is to release my new Christmas single 'BEHOLD,'" shares Phil Wickham. "I love how this song balances both new and familiar Christmas lyrics, and I can’t wait to sing this one with the Church. Thanks to the very talented Anne Wilson for joining me on it!"

Josh Baldwin | I See The Light

What joy can we not feel when reminded of the new life that comes when we put our life in Jesus’ hands! While His sacrifice may seem overwhelming and incomprehensible, He desires for us to know and feel His love. "I See The Light" takes us to a place of joy and remembrance for all He has done for us.

Tasha Cobbs Leonard | The Church I Grew Up In

"'The Church I Grew Up In' tells the story of this album. It’s what the whole record is all about. We had footage playing behind us of the very first church my father built back in my hometown. The song starts off by saying, 'There’s a little white church in Georgia with the steeple still attached,' and there wasn’t a dry eye in the house. It’s a revival song. It’s a reminder that who we are matters, that the Church matters, that the Church is still alive. I think it’s important to remind people of that coming out of this pandemic. Even though our faith may have been shaken, God’s Church is still alive and His power is still here. The song captures everything I want this album to say."

+ MORE NEW RELEASES:

Cody Carnes | Ain't Nobody

Leanna Crawford | Before I Knew Jesus

Mosaic MSC | Thanks- EP

SEU Worship | Pour It Out

 

MORE FROM MULTITRACKS.COM

APPLE MUSIC + SPOTIFY PLAYLISTS:

 

Back to top