There is no pattern that I could tell, but it has been quite a few years since I shot it.
Might be time to try again. You could run your rifle through a sequential checkout process to see what's up w/it. If it has been a few years, memory and actual fact may be out of synch.
Here's a suggested approach:
1) Systematically test the rifle by shooting 3-5 round test patterns from a good, solid bench at the 100-yard range with various kinds of ammo (mfgrs., bullet weights, powders, powder loads, COALs, etc.) to determine exactly what happens under exactly what conditions.
2) Keep detailed notes of the results.
3) Evaluate the results obtained from the above testing process.
If it turns out that the barrel shoots good groups under some of the test conditions, then assess whether these groups are good enough for your needs. For example, hunting in woods and brushy areas generally results in kill shots less than 100-125 yards. If you hunt antelope, different story.
You might try to find someone with a borescope to check the bore out for flaking, rifling issues, erosion at the leade, etc. And check headspace and the crown condition (per bkoenig) as well.
If after all this evaluation process still convinces you to replace the barrel, a new factory barrel might be a good option. Worked for you before.
Remington has a Custom Shop listed on its website. You can ship the rifle back to the factory for repairs without going through the FFL process. A Remington factory barrel might be the most value-filled option. Just call the factory first to discuss costs and make shipping arrangements.
If you want a non-factory replacement barrel, then it's a matter of carefully and systematically shopping around for the best combination of service, accuracy, and cost. As you're doing here on the Forum. Would be a good idea to do a thorough web search on any and all barrelsmiths to see what previous customers say about their work. You never, never know until you get the reworked gun home and try it out.
Folks have been burned. Even by highly recommended sources.
That's enuf for now.