The formation of patterns in two transverse dimensions in photorefractive two-wave mixing with a single feedback mirror is investigated theoretically. We perform numerical simulations of the full (3+1)-dimensional nonlinear model equations, displaying the breakup of the unstable annulus of active modes into hexagonal spots. Analytically we derive amplitude equations of the Landau type for patterns with rhombic- and hexagonal-mode interaction and discuss the stability and coexistence of transverse planforms in the photorefractive feedback system. A strong renormalization for the hexagon amplitude is determined, and its consequences for pattern formation using Landau formalism are discussed. In particular, the stability of regular substructures of a dodecagonal spot arrangement is investigated and square-hexagon competition is predicted. We use an invasive Fourier-filtering technique for the selection of unstable patterns, such as stripes and squares. The longitudinal propagation of the critical and higher-order modes of the self-organized structures and the impact of a Fourier filter on the mode propagation within a nonlinear bulk photorefractive medium is studied in detail.

• Received 22 June 2001

DOI:https://doi.org/10.1103/PhysRevA.64.063809