TLDR: if you are dealing with your own application, many issues should have a higher priority than this; if you are dealing with security auditors, yeah, most likely you will need SSL cert for ALB.
It's up to what kind of risk you can accept. Some government projects don't allow any risk from internet, so they are in intranet.
Before we start, you should create the security group on your ALB to only accept CloudFront Edge IP. You can get the ip ranges from here: https://ip-ranges.amazonaws.com/ip-ranges.json
AWS also published one guide previously, https://aws.amazon.com/blogs/security/how-to-automatically-update-your-security-groups-for-amazon-cloudfront-and-aws-waf-by-using-aws-lambda/
Back to the topic, we first need to know what kind of risk there before we decide whether it's a concern.
One legit concern is what if someone sniff my HTTP traffic between Cloudfront and ALB.
Below are quoted from AWS.
CloudFront maintains a pool of persistent connections to the origin, thus reducing the overhead of repeatedly establishing new connections to the origin. Over these connections, traffic between CloudFront and AWS origins are routed over a private backbone network for reliability and performance. This reduces overall latency for serving both static and dynamic content.
https://aws.amazon.com/blogs/networking-and-content-delivery/dynamic-whole-site-delivery-with-amazon-cloudfront/
So some will ask what does "private backbone network" mean?
Traffic traverse our infrastructure rather than the internet
Slide 23, https://www.slideshare.net/AmazonWebServices/behind-the-scenes-exploring-the-aws-global-network-net305-aws-reinvent-2018
All data flowing across the AWS global network that interconnects our datacenters and Regions is automatically encrypted at the physical layer before it leaves our secured facilities.
https://aws.amazon.com/about-aws/global-infrastructure/
So if AWS doesn't mess up, the traffic between cloudfront edge location and ALB servers are secured. It's within AWS private network backbone and encrypted before leaving AWS secured facilities.
If AWS messed up, yeah, i think you will have a much bigger problem than this SSL issue.