A computing apparatus for performing index coding and beamforming optimization between wireless devices according to an embodiment comprises a memory in which an optimization program is stored and a processor configured to execute the optimization program, wherein the optimization program inputs state information of wireless devices to a reinforcement learning model to determine an index coding action and a beamforming design action that minimize transmission time between the wireless devices. The reinforcement learning model comprises a first-level agent configured to determine, as the index coding action, whether each wireless device performs index coding and a type of an index coding scheme to be used for index coding, and a second-level agent configured to determine, as the beamforming design action, an optimal beamformer design for each wireless device in consideration of the index coding action determined by the first-level agent.