Was doing some research and found out that some players were also dealing with this issue but has not yet been resolved. Everything was running perfectly fine until after Day 18-20 or just after expanding and/or having more buildings, crops, etc.
Original Thread I found was here:
CPU: i7-4820k 3.7GHz
GPU: GTX 970
RAM: 8.0 GB
Version: Alpha 10_5-10 (64x bit)