In this paper, we propose an inference procedure for deep convolutional neural networks (CNNs) where partial evidence might be available during inference. We introduce a general feedback-based propagation approach (feedbackprop) that allows us to boost the prediction accuracy of an existing CNN model for an arbitrary set of unknown image labels when a non-overlapping arbitrary set of labels is known. We show that existing models trained in a multilabel or multi-task setting can readily take advantage of feedback-prop without any retraining or fine-tuning. This inference procedure also enables us to evaluate empirically various CNN architectures for the intermediate layers with the most information sharing with respect to target outputs. Our feedback-prop inference procedure is general, simple, reliable, and works on different challenging visual recognition tasks. We present two variants of feedback-prop based on layer-wise, and residual iterative updates. We peform evaluations in several tasks involving multiple simultaneous predictions and show that feedback-prop is effective in all of them. In summary, our experiments show a previously unreported and interesting dynamic property of deep CNNs, and presents a technical approach that takes advantage of this property for inference under partial evidence for general visual recognition tasks.