Mats: Strictly by color. It makes it much to design with and sell fabric mats, textures and other specialties. I've worked at stores where the specialties were separated out, and sure enough there would be customers who would see me reaching for fabric and tell me to just stick with "the cheaper stuff." Of course some customers still don't want to pay for them, but at least I have the opportunity to design based on design, not price.
Moulding: Combination of a couple things, but mostly by finish and design. For instance, light medium and dark woods. Within each category they are arranged from simple profiles to more ornate. Bright silver to darker silver, simple to ornate.
Some of the mouldings right behind the design area, in direct sight line to the customer, are some top-end mouldings arranged by line. The eye candy type of mouldings that catch your eye. Finally, unique and very special mouldings arranged in separate groups, things like closed-corner frames and the like.
Since customers shop based on finish and design, I think it makes most sense. It makes it easy for both the designer and customer to find what they want.
Personally I see no value in arranging moulding (or mats) by manufacturer. That's done for the framer's convenience, not to make it easier for the customer.