Physical connection is not usable when working with SqlDataReader and BLOBs

Question

I'm trying to read BLOB data from an SQL table into variables using the SqlDataReader.

The strange thing is, this exception occurs every time after about 2800 rows. I've already excluded the row in question, but apparently this is not due to the row itself, rather than the overall amount of rows (or bytes).

The exception:

A transport-level error has occurred when receiving results from the server. (provider: Session Provider, error: 19 - Physical connection is not usable)

My code:

string command = string.Format("SELECT * FROM (select *, ntile({0}) over(order by Id) as SplitNumber from BLOB) T1 WHERE SplitNumber = {1}", _maxThreadCount - (Environment.ProcessorCount * 2) + 1, splitNumber);

        using (SqlConnection sqlConnection = new SqlConnection(SourceConnectionString))
        {
            sqlConnection.Open();

            using (SqlCommand sqlCommand = new SqlCommand(command, sqlConnection))
            {
                using (SqlDataReader sqlDataReader = sqlCommand.ExecuteReader())
                {
                    if (sqlDataReader.HasRows)
                    {                            
                        while (sqlDataReader.Read())
                        {
                            try
                            {
                                byte[] blob = (byte[])sqlDataReader["Blob"];

                        [.......]

The exception occurs exactly on byte[] blob = (byte[])sqlDataReader["Blob"]; and I can't imagine why this is happening.

Any thoughts?